Introducing App::Link/XLink

Here's the place for discussion related to coding in FreeCAD, C++ or Python. Design, interfaces and structures.
realthunder
Posts: 1351
Joined: Tue Jan 03, 2017 10:55 am

Introducing App::Link/XLink

Postby realthunder » Thu Mar 30, 2017 9:58 am

Hello all. I am here to introduce a new feature named App::Link and App::XLink. You can try out my branch at https://github.com/realthunder/FreeCAD/tree/Link. Please try out the branch in debug build and report any problem you have.

These two features are an attempt to introduce a more efficient way of cloning objects in FC for 3D presentation purpose. Yes, for presentation only, not geometry modeling, which may be able get supported in future development if it is feasible at all. App::Link enables cheap clone of the 3D representation (Coin3D nodes) of any DocumentObject within the same document, while XLink supports cross document link. XLink will not actively load any external referenced document, but will monitor document opening and closing, and will update its visual if the referred object is found/lost. Both Link and XLink are extremely cheap in storage, which are essentially just text references to document and object names.

The actual heavy load lifting is done by Gui::ViewProviderLink. Link and XLink are just demonstration of its effectiveness. Any object with a PropertyLink called 'Source' can use ViewProviderLink for in-document linking, and PropertyString named 'SourceFile' and 'ObjectName' for cross-document linking. ViewProviderLink will replace the Placement property of its linked object, which enables independent object positioning. It will include the linked object name (and optionally document name if it's in another document) when selecting/pre-selecting object/subelements. The link has shared DisplayMode of its linked object, meaning that if you changed DisplayMode (or any other ViewProvider property for that matter) in linked object, the visual of all links will be updated simultaneously.

Here is piece of code for demo

Code: Select all

import Part
doc = FreeCAD.newDocument()
box = doc.addObject('Part::Feature','box')
box.Shape = Part.makeBox(10,10,10)
part = doc.addObject('App::Part','part')
part.addObject(box)
part.Placement.Base.y += 20
for i in range(4):
    link = doc.addObject('App::Link','link')
    link.Placement.Base.x += i*12
    link.Source = part
    link.ViewObject.Visibility = True
doc.recompute()
doc.saveAs('link.fcstd')

doc = FreeCAD.newDocument()
xlink = doc.addObject('App::XLink','xlink')
xlink.SourceFile = 'link.fcstd'
xlink.ObjectName = 'part'
xlink.ViewObject.Visibility = True
doc.recompute()
doc.saveAs('xlink.fcstd')


A few notes of the implementation.

FreeCAD relies on Coin3D/OpenInventor for 3D presentation. The scene graph is organised using trees of nodes. Each nodes can be a property node, like color, transformation, or shape node. Coin3D conveniently supports node sharing, which means the same shape node can be added to multiple parents to be rendered in different colors, positions, etc. This is the foundation of what make ViewProviderLink works. I think FreeCAD's design doesn't have node sharing in mind, which causes a few challenges during implementation. First, there is no enforced subtree organization of the ViewProvider. There is a somewhat template, but derived classes are free to override, which makes ViewProviderLink possible in the first place, but also makes node sharing a bit unpredictable. The assumption of ViewProviderLink is, that there is a fixed root node, a direct child SoTransform node to handle the placement, and a SoSwitch child node to hold different display mode of the actual shape nodes. It is further complicated by the special ViewProviderGeometryGroup, which directly groups other child object's root node under its SoSwith node.

The other big challenge is how to handle selection correctly. There are two ways of handling selection/preselection in FC. For OCC based shapes, FC uses a single SoFCUnifiedSelection root node to handle picking. And send action to the picked node to let it handle its own selection state. The problem is that now the node can be added to different parents, so there must be a way to store multiple selection state. ViewProviderLink uses a specilized SoFCSelectionRoot node as a selection context storage for other shape node to get/set selection state information. This mechanism works in most of the time, but have a few glitches, because of the way Coin3D handles render cache, which I don't fully understand. For now, I took the simplest approach, by touching the picked shape node, which may effectively force redraw all appearance of that shape node. I don't know if there is any way to trigger cache update based on traversal path in Coin3D. OpenInventor gurus feel free to comment. During render cache building stage, it seems the selection state information may leak to other link instance node. The other way of selection handling in FC, is for the view provider to use SoFCSelection node to handle its own selection. I think this is obsoleted, and can be converted to the new selection model, so only limited support is offered in ViewProviderLink. This affects objects like VRML, mesh, etc. Also a new function, getElementPicked, is added to ViewProvider to give derived class the full path of the picked node to resolve linked object. And once again, special handling is required for ViewProviderGeometryGroup.

A side note. Previously, an object can only appear inside one App::Part (which uses view provider derived from ViewProviderGeometryGroup). This is precisely because there is no straight forward way of handling selection resolution of its children. Now, with ViewProviderLink, you can add as many links to the same object as you want to App::Part. You can create links to an App::Part, and added them to other App::Part, as well. And also, with the capability to cross link between documents, a whole new world is opening. That is, If everything works well, of course. This is just an exploratory attempt at the moment.
Try Assembly3 (latest version 0.10.2) along with my custom build of FreeCAD at here.
And if you'd like to show your support, you can donate through patreon, liberapay, or paypal
User avatar
DeepSOIC
Posts: 7292
Joined: Fri Aug 29, 2014 12:45 am
Location: Saint-Petersburg, Russia

Re: Introducing App::Link/XLink

Postby DeepSOIC » Thu Mar 30, 2017 10:45 am

Awesome, I'll give it a try this evening, hopefully.
User avatar
yorik
Site Admin
Posts: 11645
Joined: Tue Feb 17, 2009 9:16 pm
Location: São Paulo, Brazil
Contact:

Re: Introducing App::Link/XLink

Postby yorik » Thu Mar 30, 2017 8:25 pm

Wow this looks impressive :o
Will test it tomorrow!
User avatar
DeepSOIC
Posts: 7292
Joined: Fri Aug 29, 2014 12:45 am
Location: Saint-Petersburg, Russia

Re: Introducing App::Link/XLink

Postby DeepSOIC » Fri Mar 31, 2017 12:34 am

I tested the App::Link, and so far I like it!
Is it only on my build, that selecting an element of a link to a box highlights the whole link object? (it does report the selected subelement as "Box.Face6", just the highlighting is unexpected)

I need more time to investigate it, but that looks like another kick-ass contribution, realthunder! ;)

PS. Looks like there is no button to create it. Would you mind if I knock one out and add it to part-o-magic?
User avatar
DeepSOIC
Posts: 7292
Joined: Fri Aug 29, 2014 12:45 am
Location: Saint-Petersburg, Russia

Re: Introducing App::Link/XLink

Postby DeepSOIC » Fri Mar 31, 2017 1:00 am

These are not exactly complaints, more like todos. I am assuming that App::Link is the assembly instance (or the prototype of it)
* Link (if it's to be used as assembly instance) should not withdraw the linked thing from tree root.
link-in-tree.png
link-in-tree.png (31.68 KiB) Viewed 1077 times
* It would be nice. The Cube selected in tree from within Links would correspondingly select top level instance. Interesting question follows, what should Spacebar do then. Should it hide the instance, or the actual cube? Or the cube should magically be hidden in all instances?

... this leads me to a thought, that Link shouldn't claim Cube as child. Instead, it should copy claimChildren from the Cube, that is, show nothing in this case. This will make more sense if you create an instance of a Part.

More input to come tomorrow, as I digest this stuff.
realthunder
Posts: 1351
Joined: Tue Jan 03, 2017 10:55 am

Re: Introducing App::Link/XLink

Postby realthunder » Fri Mar 31, 2017 3:33 am

DeepSOIC wrote:These are not exactly complaints, more like todos. I am assuming that App::Link is the assembly instance (or the prototype of it)
Well, this is a thread asking for comments, so feel free to spill your thoughts. This new link thing is indeed quite confusing even for me. For the picture you posted, I actually intend to forbid link of link, because that doesn't add anything except more confusion. It makes sense to have link to parts containing more links.
DeepSOIC wrote:* Link (if it's to be used as assembly instance) should not withdraw the linked thing from tree root.
Agreed. I thought about that, too. That need modification in Tree View. I plan to add the behavior later.
DeepSOIC wrote:* It would be nice. The Cube selected in tree from within Links would correspondingly select top level instance. Interesting question follows, what should Spacebar do then. Should it hide the instance, or the actual cube? Or the cube should magically be hidden in all instances?

... this leads me to a thought, that Link shouldn't claim Cube as child. Instead, it should copy claimChildren from the Cube, that is, show nothing in this case. This will make more sense if you create an instance of a Part.

More input to come tomorrow, as I digest this stuff.
In my opinion, the current behavior of App::Part is where the confusion comes from. The link behave more like other Geometry object. When claiming a child, it creates a seemingly separate identify, meaning that you can hide/show and reposition itself independently from the child. But App::Part is different. Once a child is claimed, it is no longer independent. The position and visibility state of the child are always superimposed by the part object. Toggling the visibility of a child object has seemly no effect if the Part itself is hidden. I know it has effect, it is just masked by App::Part. So, you have two behaviors, but the same Tree View representation. The confusion gets worse when you nest Parts and Links, and try to manipulate their child objects in the Tree hierarchy.

My opinion is to modify App::Part to make it behave the same as the rest, but haven't got any good idea of how to achieve the thing it can do now, like change the visibility/placement state of its children, yet still let the children manipulate their own state.
Try Assembly3 (latest version 0.10.2) along with my custom build of FreeCAD at here.
And if you'd like to show your support, you can donate through patreon, liberapay, or paypal
realthunder
Posts: 1351
Joined: Tue Jan 03, 2017 10:55 am

Re: Introducing App::Link/XLink

Postby realthunder » Fri Mar 31, 2017 3:41 am

DeepSOIC wrote:Is it only on my build, that selecting an element of a link to a box highlights the whole link object? (it does report the selected subelement as "Box.Face6", just the highlighting is unexpected)
The selection should ideally just behave as one would expect. Maybe you can post a gif or something to show the behavior?
DeepSOIC wrote:PS. Looks like there is no button to create it. Would you mind if I knock one out and add it to part-o-magic?
That'll be nice :D. On the other hand, App::Link is meant to be a quick demo only. The real use case I envisioned will be to let scripts use their own objects to implement their own logic (such as forbid link to link). The only requirement for the object is to have a PropertyLink named 'Source'. I may need to work a bit more on the extendability of ViewProviderLink, such as allow a ViewProviderExtension to control child claiming behavior.
Try Assembly3 (latest version 0.10.2) along with my custom build of FreeCAD at here.
And if you'd like to show your support, you can donate through patreon, liberapay, or paypal
User avatar
yorik
Site Admin
Posts: 11645
Joined: Tue Feb 17, 2009 9:16 pm
Location: São Paulo, Brazil
Contact:

Re: Introducing App::Link/XLink

Postby yorik » Fri Mar 31, 2017 3:49 pm

Gave it a little test, this is definitely interesting! It needs more polish of course (easy to crash, etc).

I think like DeepSOIC: The linked object should not be claimed in the tree, because it gives the message that the original object is "owned" by the copy, which is not the case, you could have several links of a same object.

There are also some face highlight problems when you do a link of an App::Part that contains, for ex, several cubes. It looks like all the Face1 of the original cubes are highlighted together. But that's a big issue that will be there when we copy a full coin node over and that will need a solution at some point. Have you tried simply copying the coin node over, without the changes you did on the slection mechanism? That worked surprisingly well in the tests I did in Arch. I didn't fully understand what your changes in the selection system do, to be honest :oops:

Also, couldn't make Xlink work.. Executing your script above doesn't show anything.

About prohibiting a link of a link, in the Draft clone I solved that by, when creating a clone of a clone, making a second clone of the base object insead. But that doesn't prevent someone to do it form python...

It certainly needs some more work, but the concept is great!
realthunder
Posts: 1351
Joined: Tue Jan 03, 2017 10:55 am

Re: Introducing App::Link/XLink

Postby realthunder » Fri Mar 31, 2017 4:06 pm

yorik wrote:There are also some face highlight problems when you do a link of an App::Part that contains, for ex, several cubes. It looks like all the Face1 of the original cubes are highlighted together. But that's a big issue that will be there when we copy a full coin node over and that will need a solution at some point. Have you tried simply copying the coin node over, without the changes you did on the slection mechanism? That worked surprisingly well in the tests I did in Arch. I didn't fully understand what your changes in the selection system do, to be honest :oops:
The whole point of this feature is to avoid copying the node. Say, we have a super complex compound object built using OCC, FC uses only one Coin3D node to represent all its faces. If we have to copy that node, then we might as well use the existing Draft Clone. Because OCC TopoDS_Shape is already using geometry object sharing under the hood, so copying is cheap. All the overhead is actually in building/updating Coin3D node.

What OS do you use? On my linux machine, the selection and XLink works fine, small glitches from time to time, but fine most of the time. I do actually worry that there are platform differences in Coin3D. Hopefully it won't undermine the whole concept.

Edit: I just tried a fresh build on Windows, it works the same as on Linux. Could you post your project file, or better, with code, since I haven't really tested the effects of file saving/restoring on links.
Last edited by realthunder on Fri Mar 31, 2017 7:43 pm, edited 1 time in total.
Try Assembly3 (latest version 0.10.2) along with my custom build of FreeCAD at here.
And if you'd like to show your support, you can donate through patreon, liberapay, or paypal
User avatar
DeepSOIC
Posts: 7292
Joined: Fri Aug 29, 2014 12:45 am
Location: Saint-Petersburg, Russia

Re: Introducing App::Link/XLink

Postby DeepSOIC » Fri Mar 31, 2017 6:06 pm

realthunder wrote:Because OCC TopoDS_Shape is already using geometry object sharing under the hood, so copying is cheap. All the overhead is actually in building/updating Coin3D node.
That's not exactly true. Draft Clone does a deep copy of the shape. In Lattice, I made array tools to do a shallow copy, to get extra performance and lower memory use. That worked brilliantly, and I got an unexpected speedup because OCC re-uses the triangulation too... to finally realize that after saving-loading, all the sharing is lost. Even the array with internal sharing, written out as compound into a brep file. The sharing goes away. That makes me wonder, how does a compsolid survive save-load, it has to share faces to be valid...