"Robust References" Redux

Here's the place for discussion related to coding in FreeCAD, C++ or Python. Design, interfaces and structures.
ezzieyguywuf
Posts: 238
Joined: Tue May 19, 2015 1:11 am

"Robust References" Redux

Postby ezzieyguywuf » Sat May 21, 2016 3:28 am

Edit: See second post for progress, current state, and next steps, as well as some links to milestone posts within this long thread.

Back in 2012, jrheinlaender started a thread whereby he suggested an approach to solving the topological naming issue that FreeCAD currently has. I revived this post a few weeks again, and have since then spent a considerable amount of time researching this topic and reading my way through both the existing FreeCAD code base (roughly version 0.17 as of the time of tis post) as well as ickby's tnaming branch in which he had begun to incorporate some of jrheinlaender's original code into the FreeCAD code base.

I am starting this new thread for a few reasons:

1) I want a place to gather some of the information I've found, to make it easier for anyone else that may join me on this journey
2) I want a place to discuss current progress (or lack thereof) on this issue
3) I want to bring my current work - limited though it is right now - to the attention of the main devs.

I find it very unfortunate that jrheinlaender made so much progress and produced so much code only to be met at the end with a bit of resistance to to his approach. In all fairness, a do agree with jriegal that this topological naming approach should not reside in the Part Design module but rather in the Part module, but I want to avoid this type of issue by discussing upfront my plans, and what I want to do.

First, a word about 'Geometry' and 'Topology' for those that, like me, are completely new to all this. As far as I can tell, 'Geometry' of a shape are points (represented by coordinates), lines (defined by two points), planes (defined by three points), and volumes (defines by...not sure) that define the shape. In other words, the typical things we're used to from Geometry class, i.e. y=mx+b to define a line and whatnot.

The 'Topology' of a shape, on the other, is distinct from the geometry in that it is more general. It refers to vertices, edges, surfaces, and solids rather than points, lines, planes, and volumes. The difference is that an edge is essentially a portion of an underlying line, a surface a portion of an underlying plane, etc...

To be fair, I don't 100% understand the difference, but I am led to believe that there is a preference to deal with 'Topology' rather than 'Geometry' when it comes to modeling, because of, like, memory usage and stuff.

That being said, here's what I know so far:

Per ickby, here is a recommended roadmap for moving forward:

ickby wrote:items in RED are not from ickby's original post

The general way for this topic undependend of the exact topological naming implementation is like that:
  1. Extend TopoShape with possibility for robust references
    1. Add data structure for saving history of elements
    2. Extend all algorithms to build up the element history
    3. Add methods to query elements robustly by some reference or by construction method
  2. Move all occ algorithms that modiy the 3D model and are in PartDesign or anywhere else in FreeCAD into TopoShape and make them work with the robust reference system
  3. Make all Workbenches use the new TopoShape Methods and hence support robust references
  4. Rewrite all Gui tools so that they build up robust references instead of the current weak once
  5. Add a Gui for the user in case references get lost so that he can assign new ones


As far as I can tell, ickby's branch that I list above has completed 1.1, begun 1.2, very minimally started 1.3, and hasn't done anything else.

I am going to try to write up a protoype in pure python, maybe python-occ, since I'm most comfortable coding in python. I won't spend too much time on this, but I figure if I can whip something together that it will help wrap my mind around what needs to happen, and I can post it here and it may spark some discussion.

Other stuff - as far as I can tell, the following is the current state of topological naming/reference in FreeCAD. Please feel free to correct me here if I'm wrong as my experience with the codebase is minimal and I may be misunderstanding things:

  1. TopoShape is the base class which is used (mostly) for interfacing with the opencascade topological libary (i.e. BRepBuilder and friends)
    1. TopoShape depends on ComplexGeoData
    2. There are currently other Modules (i.e. Part Design) that do not do all of their opencascade interface through TopoShape, hence ickby's item 2 above
    3. TopoShape includes methods such as 'makePipe', 'cut' (booleon substract), 'fuse' (boolean add), 'transformGeometry', and notably 'getSubShape'
  2. As far as I can tell, Modules such as 'Part Design' are intended to use TopoShape as follows:
    1. User creates a geometry (i.e. a closed loop of lines)
    2. Module uses geometry to create a Topological Shape using TopoShape
    3. User modifies geometry (i.e. extrudes sketch or creates pocket in solid)
    4. Module uses relevant TopoShape method. In the case of extrude, this would be 'makePrism' I think. In the case of pocket, it would be 'cut' I think. Of note is that the 'cut' method accepts a geometry which is used to operate on the existing TopoShape. In other words, a shape (let's say a cylinder) is passed into the 'cut' metod of an existing TopoShape, let's say it's a cube. TopoShape then subtracts the cylinder from the cube and creates an entirely new Topological Shape.
    5. Module saves the resulting Topological Shape as the new shape that the user interacts with
  3. As far as 'Topological Naming', the way this is currently done is that the opencascade class 'TopExp_Explorer' 'TopExp::MapShapes' is used to 'explore' the Topological Structer stored in TopoShape. As the TopoShape is explored, each edge and vertex is assigned an increasing number, i.e. 'Edge1', 'Edge2', etc...
  4. When the Module requires to perform an action on a particular Topological Entity (i.e., 'Face1' on a cube), it specifies the name, literally 'Face1', and then TopoShape iterates over the Topological Face until it finds a 'Face' that has index 1. I'm a bit confused as to how the Module currently keeps track of which face is which, but suffice to say this is the root of the Topological Naming problem

As far a a future state, here's what I believe the intended goal is, and what ickby/jreinheilander have started coding:
  1. TopoShape depends on ComplexGeoData as above
  2. ALL topological shapes are managed through TopoShape - no other modules make calls to 'BRepBuilderApi' from opencascade
  3. As methods are called on TopoShape, i.e. 'cut', 'makePrism', 'fuse' and friends, TopoShape keeps track of the changes
  4. By keeping track of these changes, a 'Robust Reference' to topological entities will be created. Rather the module referring to 'Face1' it will refer to 'The Face resulting from the Edge that is between the two Vertexes that are furthest from the origin in the Shape that was passed to the consrtuctor'. Of course, the syntax would be more concise,
  5. Anytime something 'catastrophic' happens, i.e. an edge wit a Fillet is deleted, the TopoShape alerts the Module - or rather, provides a method that allows the Module to check before making changes - so that the Module can alert the user

I'm still trying to understand ickby/jrheinlaender's code, but it seems that so far it's keeping track of some sort of '_History' in the TopoShape class. I don't think the actual 'Robust Reference' portion of it has been implemented yet. Based on my research, I think than this and this reference, both from the Catia documentation, do a pretty good job of explaining an approach that could work.

This is all I have so far! Please, help me to fill in the gaps. Like I said, I'm still working on reading code and understanding things. But this is a big project, and in all my years of working on projects (both academically and professionally), the bigger the project, the more time that must be spent planning in order to avoid failure. Thus, I'm trying to spend an appropriate amount of time on recon and planning before diving into contributing more code.
Last edited by ezzieyguywuf on Wed Jun 08, 2016 1:49 am, edited 4 times in total.
ezzieyguywuf
Posts: 238
Joined: Tue May 19, 2015 1:11 am

Re: "Robust References" Redux

Postby ezzieyguywuf » Sat May 21, 2016 3:34 am

Progress so far.

NOTE: Between update 13 and 14 I changed the naming of my github repo branches. I may or may not have updated some of the links in this thread. The original 'tnaming_ezziey' branch is now 'tnaming_eziey_00'

1) May 2016, reviewed current state and previous attempts. see first post in this thread
2) May 2016, Compiled opencascade qt sample and pythonocc. see this post for more info
3) May 2016, duplicated topo naming issue in opencascade. see this post for more info
4) June 2016, Compiled simple opencascade program from scratch. see this thread.
5) June 04, 2016, Finished extending opencascade program to duplicate work done in python-occ. see this post for more info and for full source code
6) June 06, 2016, Posted github repo with working example of TNaming showing how to recover a TopoDS_Shape after it has been modified without referring to its Index
7) June 07, 2016, Posted writeup of how TNaming works and what we need to do to make it work. See this post for a TL;DR and a link to the full writeup.
8) June 09, 2016, Posted an update to github with 'helper functions' for Data Framework transactions. See this post for more details.
9) June 09, 2016, Posted a 'shape dump' that show the Fillet Bug (linked below) resolved with TNaming. See this post
10) June 21, 2016, Posted an update with the begginings of FreeCAD integration. See this post.
11) July 08, 2016, Posted an update with some refactored FreeCAD integration. See this post. A cube fillet and rebuild now works without storing the edge as an int but rather by using TNaming.
12) July 25, 2016, Posted an update with more refactored FreeCAD integration. See this post. I've implemented some of ickby's recommendation from the conversation in this thread.
13) July 27, 2016, Posted a minor update here. I've updated the 'TNaming Writeup' document to include info about 'TNaming_Selector'
14) August 19, 2016, Posted an updated with refactored TopoNamingHelper as well as re-organized git repo. See this post.
15) August 26th, 2016, Posted an update that shows a partial fix for the fillet bug. See this post for more info. This is a huge milestone!

Next step:
- [Done!] duplicate topo naming issue using just occ.
- [Done!]Try to use occ TNaming to obtain a constant reference to an Face, regardless of other operations performed on the TopoDS_Face
- [Done!] Incorporate the Fillet stuff from the opencascade TNaming tutorial into our little sample
- [Done! mostly...may need to update with Selector stuff]Write up a brief (or not so brief) explanation of how this TNaming stuff works, and what is required in order to make it work (i.e. tracking Modified, Deleted, and so forth TopoDS_Shapes). Make sure we understand fully how to use TNaming (i.e. what a Selector is and when to use it)
- [Done!] Clean up code a bit
- [Done!]Create some more test-cases other than Fillet and Cut to test the TNaming solution again - perhaps that Gear thing that someone posted on here? Update: Will try to duplicate this bug and resolve it with TNaming
- Write up a high-level overview for incorporating TNaming into FreeCAD, i.e. how we will keep track of the occ Data Framework, how we will add data to the Data Framework, how we'll keep track of Labels, etc
- [half done, have not resolved bug yet] Start FreeCAD branch with very basic TNaming incorporated, enough to replicate this issue and resolve it
- [paused work] Figure out how to Serialize and DeSerialize the Data Framework, so that we can persist it across sessions
- Figure how, or if, we can join two Data Frameworks together
- Fix hard-coded TopoNamingHelper::TrackModifiedShape code, which right now was hacked as a proof-of-concept. Needs to work for any generalized case, i.e. needs to somehow determine Modified/Generated/Deleted Faces between Base Shape and Modified Shape.
Last edited by ezzieyguywuf on Sat Aug 27, 2016 12:06 am, edited 21 times in total.
wandererfan
Posts: 886
Joined: Tue Nov 06, 2012 5:42 pm

Re: "Robust References" Redux

Postby wandererfan » Sat May 21, 2016 12:53 pm

2. Move all occ algorithms that are in PartDesign or anywhere else in FreeCAD into TopoShape and make them work with the robust reference system


This one will have to be reworded. We use occ algorithms (mostly HLR, but also distances and intersections, etc) a fair bit in TechDraw (and Drawing). Maybe "occ algorithms that modify the 3D model"?

We also use weak references. but we have a Gui function to change them.

wf
User avatar
tanderson69
Posts: 1291
Joined: Thu Feb 18, 2010 1:07 am

Re: "Robust References" Redux

Postby tanderson69 » Sat May 21, 2016 3:21 pm

The occt topo naming is built around the virtual functions of BRepBuilderApi_MakeShape: Generated, Modified and IsDeleted. Once you start examining the output of these objects, you will see they are incomplete. Most output involves faces and ignores the edges and vertices. I believe that is by design. If you read the link provided below, it mentions opencascade and talks about 'tracking' faces and using the faces to then map to edges and vertices. My observations from the output of BRepBuilderApi_MakeShape and reading the paper lead me to believe that this paper is the foundation for the toponaming currently implemented in occt. http://www.lias-lab.fr/publications/7076/2003-ICSMA-AGBODAN.pdf


Just in case you missed it.
http://www.freecadweb.org/wiki/index.php?title=Naming_project
ezzieyguywuf
Posts: 238
Joined: Tue May 19, 2015 1:11 am

Re: "Robust References" Redux

Postby ezzieyguywuf » Sun May 22, 2016 12:31 am

wandererfan wrote: ...This one will have to be reworded. We use occ algorithms (mostly HLR, but also distances and intersections, etc) a fair bit in TechDraw (and Drawing). Maybe "occ algorithms that modify the 3D model"?...


Done, made the changes in red.

tanderson wrote:Just in case you missed it.


Nope, didn't miss it. Thanks for the link though. Should I move the bulk of my OP to the wiki? Is that a more fitting place? Or is the forum ok? It seems that the forum gets more foot traffic than the wiki does...

Regarding the article you posted, that's a great resource! I've read through most of it so far and it makes a lot of sense. I'm going to spend some time tinkering with 'BRepBuilderApi_MakeShape' as you noted, to get the lay of the land. When you mention 'examining the output of these objects' are you doing this in FreeCAD? Or are you building minimal occ programs and doing that way? I'm thinking about using pythonocc to do the tinkering, as it seems like a pretty straightforward wrapper around opencascade itself.

Finally, on a related note, I was reading through the changelog of OpenCascade 7 and saw some mention about Topological Naming. It seems they've added a section to the documentation that explains the TNaming stuff a little bit. It can be found here in the OCAF section of the user guides. I'm reading through this now, I'm hoping that this may be the silver bullet to the whole topological naming thing. *crosses fingers*
Last edited by ezzieyguywuf on Sun May 22, 2016 12:26 pm, edited 4 times in total.
ickby
Posts: 2426
Joined: Wed Oct 05, 2011 7:36 am

Re: "Robust References" Redux

Postby ickby » Sun May 22, 2016 5:58 am

Very nice wrap up!

ezzieyguywuf wrote:I find it very unfortunate that jrheinlaender made so much progress and produced so much code only to be met at the end with a bit of resistance to to his approach. ... I want to avoid this type of issue by discussing upfront my plans, and what I want to do.

Yes, this is IMHO very important for a Topic that deep in FreeCADs code heart. If you take us with you forward step by step the result will be easy incorporated.

I am going to try to write up a protoype in pure python, maybe python-occ, since I'm most comfortable coding in python. I won't spend too much time on this, but I figure if I can whip something together that it will help wrap my mind around what needs to happen, and I can post it here and it may spark some discussion.

Taht is a very good idea, using python for rapid prototyping data structures and algorithms will make it way faster to come up with and share ideas. This makes it easy for us to check out things you created by simply running scripts.

Your list is correct in nearly all points. One small addition:
I'm a bit confused as to how the Module currently keeps track of which face is which, but suffice to say this is the root of the Topological Naming problem

The links to faces/edge etc. are stored in PropertyLink properties as strings, the subobject of those links is "Face1" or "Edge2". Than you will find methods in TopoShape to extract subshapes from those strings. The method simply extracts shapetype and number and gets the subshape, e.g. "Face1" builds a TopExplorer for Faces and gets the first face out of it.
jmaustpc
Posts: 8074
Joined: Tue Jul 26, 2011 6:28 am
Location: Australia

Re: "Robust References" Redux

Postby jmaustpc » Sun May 22, 2016 2:21 pm

ezzieyguywuf wrote: I'm thinking about using pythonocc

I think you will find that pythonocc only supports rather old versions of OCE, at least not the latest OCC. The last time I looked they still only had OCE 0.17.1 support in a review branch.
User avatar
tanderson69
Posts: 1291
Joined: Thu Feb 18, 2010 1:07 am

Re: "Robust References" Redux

Postby tanderson69 » Sun May 22, 2016 2:40 pm

ickby wrote:The links to faces/edge etc. are stored in PropertyLink properties as strings, the subobject of those links is "Face1" or "Edge2". Than you will find methods in TopoShape to extract subshapes from those strings. The method simply extracts shapetype and number and gets the subshape, e.g. "Face1" builds a TopExplorer for Faces and gets the first face out of it.


Sorry to 'nit pick', but TopoShape uses TopExp::MapShapes not TopExplorer. TopExplorer will revist the same sub shape multiple times.
https://github.com/FreeCAD/FreeCAD/blob/739509aadc5f155bcc3c44bc9b324929c6163e35/src/Mod/Part/App/TopoShape.cpp#L377
ezzieyguywuf
Posts: 238
Joined: Tue May 19, 2015 1:11 am

Re: "Robust References" Redux

Postby ezzieyguywuf » Mon May 23, 2016 2:34 am

ickby wrote:Taht is a very good idea, using python for rapid prototyping data structures and algorithms will make it way faster to come up with and share ideas. This makes it easy for us to check out things you created by simply running scripts.


Ok great! I'll spend some time trying to hobble something together for prototyping then.

ickby wrote:The links to faces/edge etc. are stored in PropertyLink properties as strings, the subobject of those links is "Face1" or "Edge2".


I did not know this. I'll take a look at PropertyLink. I was aware of the TopoShape methods that breaks up a std::string into, say, ['Face', 1], or ['Face', 2], etc... and then used that to find the relevant face in the Shape. My real question was: how does, for example, Part design know which Face to ask for? Without diving into the code too much further, I assume it has something to do with Selection (which is, itself, it's own class somewhere, right?). My guess is that the Selection thing allows the user to select a feature by taking the current coordinate system orientation and highlighting the Shape that is closest "to the screen" underneath the mouse. In other words, if the user has a "Top Down" view, then the Shape with the highest Z-coordinate is what would be highlighted, and at this point TopoShape would provide Part Design with the face number of that Shape. That's just my guess...

jmaustpc wrote:I think you will find that pythonocc only supports rather old versions of OCE, at least not the latest OCC. The last time I looked they still only had OCE 0.17.1 support in a review branch.


Yup, I was aware of this. I figured it may still be helpful for prototyping purposes though. To be honest, after some headaches I finally got pythonocc installed and working, but so far it's not been terribly useful. Maybe I just need to spend some time with it, but maybe that time would be better spent elsewhere...

tanderson69 wrote:Sorry to 'nit pick', but TopoShape uses TopExp::MapShapes not TopExplorer. TopExplorer will revist the same sub shape multiple times.


No need to apologize, these are just the types of clarifications I was hoping for. Thanks!
ezzieyguywuf
Posts: 238
Joined: Tue May 19, 2015 1:11 am

Re: "Robust References" Redux

Postby ezzieyguywuf » Sat May 28, 2016 12:56 am

Current status: So I've successfully compiled the opencascade Qt Tutorial, as well as python-occ. The Qt tutorial is for version 7 of opencascade, and the python-occ is for an older version (not quite sure which).

Why'd I do this? Well, I want to get a better understanding of the underlying mechanics of how opencascade names things, and it was getting a bit cumbersome doing so within FreeCAD (add Base::Console().Log() things, recompile, wait, see crash, repeat) so I wanted to find a lighter-weight way of doing this. I started with python-occ but got frustrated when I couldn't loop over a TopTools_IndexedMapOfShape, so that's why I decided to try the opencascade Qt Tutorial.

When I got the Tutorial running, though, I realized that my C-foo is rather limited, and it seemed daunting to add enough to the base Tutorial to do what I want, so I went back to python-occ.

I have figured out how to do things in python-occ: it's basically doing things in C, but with some handy python-like ways of doing things (like using print).

I am left with a question though: how does the current-state FreeCAD determine which of a feature has been selected? In other words, to keep the example simple, if I have a cube and select a face in the GUI, how does FreeCAD turn this into 'myIndexedMapOfShapes.FindKey(KEY)'. i.e., how does the GUI determine which KEY to send to the FindKey method?

My next step is to duplicate the issue that we have with Topological Naming in python-occ. But right now, I have a cube, I have an IndexedMapOfShape, but I don't know which face is which. i.e. I don't know which is my Bottom face, which is my Top, etc. so I have no point of reference for whether or not the index changes when I, say, create a pocket.