Preview: Link, stage two, API groundwork

Post here if you have re-based and finalised code to integrate into master, which was discussed, agreed to and tested in other forums. You can also submit your PR directly on github.
realthunder
Posts: 1028
Joined: Tue Jan 03, 2017 10:55 am

Preview: Link, stage two, API groundwork

Post by realthunder » Fri Jul 28, 2017 6:48 pm

This is the second stage of introducing the Link concept into FC. The first stage pull request is here. I haven't submit it as PR yet, because the first one is still pending. I bring out this early as a preview, so that we can have some discussion about how FC shall handle local coordinates in the future. I had some discussion (here, and here) with @ickby before, but got no where. I am hoping to reach some consensus here by discussing over something concrete.

The patch set of this stage introduced quite a few new core APIs to build the foundation for the Link concept, and at the same time, as a solution for handling of various local coordinates. Everything added is callable and can be overridden in Python, which makes it possible to create fully functional Link object in script, as is shown in the attached test script. It is not trivial to implement the link right, which is why a C++ built-in feature, App::Link, will be added in the upcoming final stage.

To test, please clone my branch at https://github.com/realthunder/FreeCAD/tree/LinkStage2. Download and put the testing script in the attachment to your Macro directory (Edit: oops, just found out that I attached wrong script. Please try download again.). You can read the script code and the comments inside, in case you are interested to find out which new APIs are responsible for the new behaviors.

The first demo continues from stage one, and shows the correct tree view behavior of a link group.

Code: Select all

import linkDemo
linkDemo.linkGroupTest()
It create a box, a cone, and a fusion (yellow) of the two, and then group these three in one group. The box is initially hidden in the group. You can use tree view to toggle its visibility as expected. Notice that when the visibility is changed, the group object is marked for recomputing. This is because the child visibility is a feature property instead of view object. And it is made so that group feature object can be notified to update its internal compound shape representation of the whole group member, which made it possible to add group to normal Part::Feature tools.

As I mentioned in the previous stage, the objects added to a LinkGroup exists both in group's local coordinate system (CS), and the global CS. All existing FC tools only works with objects in global CS. So, to show the object in global CS, expand the fusion object, and toggle its child's visibility, you can see the object at a different placement. If, however, there is no FC tool claiming a member object, you can't access the group's children in the global CS using tree view, which is why I added a new API to let object control whether to remove the claimed children from tree root. Toggle group object's 'ChildAtRoot' property, you can reveal the fusion object in global CS at the tree root. The other members of the group, box and cone, are not shown at root because they have been claimed by fusion, which has the default remove-children-from-root behavior. Move the fusion object in the global CS also moves the one in the local coordinate, as expected. Moving the group object only affects the object inside group, not those at the global CS.

Because now one object can potentially be inside a lot of different groups, hence, different at coordinates, tree view has a convenience menu action to select all appearances of the objects in the entire application (you will soon find out that an object can now appear in multiple documents).
Image


LinkGroup uses a different approach from GeoGroup to implement the group behavior. It uses Coin3D node sharing for visual representation as mentioned in stage one. The current stage introduced a new API, DocumentObject.getSubObject(), to impelement the non-visual behavior of a group with local coordinate system. Gui.Selection has been modified to accept a new parameter 'resolve' (defaults to True) in some of its API (getSelection(), getSelectionEx(), getCompleteSelection(), getTypeOfObjects(), addSelectionGate(), etc), which calls getSubObject() to resolve the object reference inside a full qualified SubName. Existing FC tools can work with objects inside a LinkGroup without modification. However, the default 'resolve' behavior is to resolve to object in global CS. You can select the box and cone inside the group object, and choose Part.Fusion tool to create a fusion object and notice the resulting fusion's placement. To work directly with objects in local coordinate, the tool needs to be modified to resolve the object manually by calling getSubObject(), which is capable of returning the full transformation matrix of the sub object. However, it is my opinion that this is an unnecessary complication, and that all existing tools shall still operate in global CS as they are now. A special Link type object can be used to link to sub object in nested local coordinates. The link itself still exists in global CS and can be operated as usual by existing tools, with minimum modification.

The testing script implements a Link type called _Link (and the upcoming stage will bring you App::Link) to do just that. The Link can link directly to an object in global CS, overriding its placement, and thus creates a separate entity in global CS. Or, it can link to a sub object inside arbitarily nested groups. It overrides the top level group's placement, but preserves and automatically sychronizes the accumulated local coordinates of the (nested) sub object. The Link is also capable of override the linked object's material. The following test sets a distinct color of each link for easy demonstration. One thing though, neither Link nor LinkGroup can work with App::Part because of App::Part's unique way of grouping children visually. The upcoming App::Link has special code to support that.

Code: Select all

linkDemo.linkTest()
The test creates a second group containing the group above. It then creates a Link which links to group2.group.box. The link itself can be moved freely without affecting others, which essentially have a local CS of its own. Moving group2 does not affect the link, either, because the link overrides the top level group's placement. Moving group2.group, however, affects the link placement as expected. The demo also shows how the link can be easily changed dynamically. It can even link into non-object sub-element, such as a Face. When using the property editor to change the link, it always refer to the object in the global CS. To link to sub object, you can use drag and drop, as shown in the following screencast. When you selects a link object, the property view shows both the property of the link and its linked object. For properties having the same name, such as 'Placement', it will be from the link, instead of the linked object.
Image


The next demo shows that Part::Feature tool can work with links. Currently, Part workbench only has partial support of link type object. Toolbar commands only works when the link actually points to a real Part::Feature. You can use non Part::Feature linked object in script, as long as the linked object has a proper implementation of getSubObject() (such as LinkGroup here) that can return Part.Shape when asked. You can trick the GUI to accept non Part::Feature links by assigning the link to a Part::Feature first, create the the tool with the link, and then change the link afterwards, as shown in the following screencast. The screencast also shows how to assign the link using GUI. You can either use property editor, which only selects objects in global coordinates, or use drag and drop which supports sub objects in local coordinates.

Code: Select all

linkDemo.linkFusion()
Image


The next demo shows the newly added PropertyXLink that can link to object outside the current document. It can be used just the same as PropertyLink. It stores additional information of the object's document path (relative to the owner document) and the document file's time stamp. As shown in the following screencast, you can simply drag the object across document boundary and drop to the link. Or, simply assign the property with any object using script. It will store the document information if the object is external. The tree view has been enhanced to show external object, and its claimed children. An overlay arrow is shown in the right bottom of the icon to indicate that it is an external object. When the external document is closed, all linked PropertyXLink automatically lose the linked object reference, but still retains the document and object name information, so that when the external document is opened again, the links can be automatically restored. The document recomputation logic is also enhanced to support external object dependency. Finally, the screencast shows that when a document containing PropertyXLink is opened, FC will automatically open all referenced external documents.

Code: Select all

linkDemo.linkXTest()
Image


The final screencast shows some of the other tree view enhancements.
* Sync Selection, when enabled, selection in 3D view will automatically select the corresponding tree item and expand its parent(s) if necessary.
* Sync View, selecting an object in the tree will automatically switch to its 3D view.
* Select all instances, select all occurance of an object, expand the parents if necessary.
* Select all links, select all links (direct or indirect) to some object.
* Hide item, hide the object in the tree view. All tree items (including externally linked) corresponding to this object will be hidden. The setting is stored into 'ShowInTree' property of the object.
* Show hidden item, reveal the hidden items. This setting is per document, and stored in the document's 'ShowHidden' property. When this setting is on, the hidden items' icons are marked by a small 'eye' overlay image in the left top corner.
Image

Edit: oops, just found out that I attached wrong script. Please try download again.
Attachments
linkDemo.py
(26.81 KiB) Downloaded 48 times
Last edited by realthunder on Mon Jul 31, 2017 8:53 am, edited 2 times in total.
Try Assembly3 (latest version 0.10.2) along with my custom build of FreeCAD at here.
And if you'd like to show your support, you can donate through patreon, liberapay, or paypal

User avatar
DeepSOIC
Posts: 6678
Joined: Fri Aug 29, 2014 12:45 am
Location: Saint-Petersburg, Russia

Re: Preview: Link, stage two, API groundwork

Post by DeepSOIC » Fri Jul 28, 2017 8:44 pm

Nice to see you are still trying to push it through! :D

I have one small question. I remember from my earlier experimentation with your branch, that you used complex subelement/child access in a form like so: "Part.Body.Pad.Face1". The question is, if I use "Part.Body.Face2", and Body happens to contain DocumentObject named Face2, what will be returned? Body.Shape.Face2, or Face2.Shape?

triplus
Posts: 8456
Joined: Mon Dec 12, 2011 4:45 pm

Re: Preview: Link, stage two, API groundwork

Post by triplus » Sat Jul 29, 2017 11:28 pm

realthunder wrote:
Fri Jul 28, 2017 6:48 pm
As I mentioned in the previous stage, the objects added to a LinkGroup exists both in group's local coordinate system (CS), and the global CS. All existing FC tools only works with objects in global CS.
Is that true for PartDesign NEXT? AFAIK no as that is basically why a lot of Part workbench commands where "forked". To take local coordinate system into account.

realthunder
Posts: 1028
Joined: Tue Jan 03, 2017 10:55 am

Re: Preview: Link, stage two, API groundwork

Post by realthunder » Sun Jul 30, 2017 3:05 am

DeepSOIC wrote:
Fri Jul 28, 2017 8:44 pm
Nice to see you are still trying to push it through! :D

I have one small question. I remember from my earlier experimentation with your branch, that you used complex subelement/child access in a form like so: "Part.Body.Pad.Face1". The question is, if I use "Part.Body.Face2", and Body happens to contain DocumentObject named Face2, what will be returned? Body.Shape.Face2, or Face2.Shape?
Body is derived from Part::Feature. And Part::Feature::getSubObject calls the parent implementation first, which calls GeoFeatureGroupExtension::extensionGetSubObject. So if there is an object named Face2, it will return that object instead of the subelement.

However, getSubObject has an option to either return Shape, or a tuple (Object,sublement,matrix,PyObj). If you only ask for Shape, which is the default behavior, then there will be ambiguity. If you ask for later, then if there is an Face2 sub object, the return tuple will have Object=Face2, subelement='', PyObj=Face2.Shape. If there is no Face2 sub object, then Object=Body, subelement='Face2', PyObj=Body.Shape.Face2. You can read the python doc string of getSubObject for more details.
Try Assembly3 (latest version 0.10.2) along with my custom build of FreeCAD at here.
And if you'd like to show your support, you can donate through patreon, liberapay, or paypal

realthunder
Posts: 1028
Joined: Tue Jan 03, 2017 10:55 am

Re: Preview: Link, stage two, API groundwork

Post by realthunder » Sun Jul 30, 2017 4:06 am

triplus wrote:
Sat Jul 29, 2017 11:28 pm
realthunder wrote:
Fri Jul 28, 2017 6:48 pm
As I mentioned in the previous stage, the objects added to a LinkGroup exists both in group's local coordinate system (CS), and the global CS. All existing FC tools only works with objects in global CS.
Is that true for PartDesign NEXT? AFAIK no as that is basically why a lot of Part workbench commands where "forked". To take local coordinate system into account.
How PartDesign works is that, every object created will be auto added to some Body container, which means it's under Body's local coordinate. And every tool using those objects will be put under the same Body container, which means the tool's output Shape will be under the same local coordinate. The underlying geometry (Shape) of objects inside Body container is not transformed, so they are still in the global coordinate. You can verify this by add a Box to Body, move the Body, and then Part.show(Box.Shape). The box shape will be at different placement. This is what I mean by 'All existing FC tools only works with objects in global CS', or more precisely, it only works with geometry data in global CS.

This is exactly my problem with geo group (@ickby), which is what Body derived from. It gives you the illusion that the tool is operating on objects in local coordinates, but in fact it is not, because both the input and output geometry data are still in global CS. And to maintain that illusion, geo group demands all objects used by the tool must be in the same geo group as the tool itself. It works fine for Body in PartDesign, because the tools of PartDesign do not claim any children, and it is implicitly assumed by the tool that all's children are inside the same Body. But, it is my believe that this restriction is going to serious limit App::Part's usefulness for assembly purpose, where you can have complex nested containers. And the main purpose of this preview is to show my way of lifting this restriction.

Put in another way, every tool in FC is in fact working with geometry data in global CS. My work is just to make sure that 3D visuals can faithfully reflect that fact. If you want to see the true relationship of a fusion object and its children object, simply toggle the visibility of its claimed children, which will be in global CS, and the fusion itself under global CS like I did in my first screencast. This relationship is fixed, no matter what container the fusion or its children belong. You can move those objects into different containers, and the relationship as shown in 3D view stays the same. This is not the case for App::Part, because objects added to it is visually removed from global CS (i.e the 3D view), and the fusion object claiming a child inside some App::Part can only be shown in its local coordinate. So the parent child relationship shown in 3D view is only correct if they are at the same container.
Try Assembly3 (latest version 0.10.2) along with my custom build of FreeCAD at here.
And if you'd like to show your support, you can donate through patreon, liberapay, or paypal

ickby
Posts: 2906
Joined: Wed Oct 05, 2011 7:36 am

Re: Preview: Link, stage two, API groundwork

Post by ickby » Sun Jul 30, 2017 7:55 am

I did not yet have time to look into the code, but I need to answer on your last posts, just because there are many faults in it.
... every tool using those objects will be put under the same Body container, which means the tool's output Shape will be under the same local coordinate. The underlying geometry (Shape) of objects inside Body container is not transformed, so they are still in the global coordinate.
That is nonsense. The whole purpose of the notion of local coordinate systems is that the things contained in it must not change. You make this mistake on other parts of you text too: You assume that just because the shape placement is not changed it is still in global coordinates. But this is totally wrong.

The shapes values, e.g. the point of a Vertex, need a reference to what those values are defined. Until 0.16, this reference has always been the global coordinate system, the one you see on the screen. Now, with geofeaturegroup, this can also be a local coordinate system. If you put the shape in a local cs it still have the exact same point values for that vertex. If you move that geofeature group, the vertex point inqueried from the shape is still the same. But it is referenced to the local cs. and that has changed. Hence calculating the point in the global CS must take both placements into account, something like: Pglobal = RlocalCS * Plocal + TlocalCS

And this works perfectly well with geofeaturegroup. When you change the geofeaturegroups placement the contained things do not change at all, but in global CS they get moved, just as they should (you can see it on the screen). This is how it shall work.
It gives you the illusion that the tool is operating on objects in local coordinates, but in fact it is not, because both the input and output geometry data are still in global CS.
Again the same mistake. You can not distuinguish from geometry alone to which CS it belongs, it is a pure definition. And we defined that it is always valid for the CS it is contained in, so in the geofeaturegroup it is contained in. Thats how we build the architecture. And frankly, this is how it must work if one wants to achieve assembly constraint functionality.
Put in another way, every tool in FC is in fact working with geometry data in global CS
Wrong. Every tool in freecad works with geometry data in a unique CS, meaning input as well as ouput must be in the same CS. It does not matter which CS this is, or how man local CS are stacked on top of each other.
My work is just to make sure that 3D visuals can faithfully reflect that fact.
Your work breaks the architecture. Everything regarding coordinate systems is a question of definition, how do we setup the math, and then making freecad work according to those definitions.

realthunder
Posts: 1028
Joined: Tue Jan 03, 2017 10:55 am

Re: Preview: Link, stage two, API groundwork

Post by realthunder » Sun Jul 30, 2017 9:57 am

Hello ickby, glad to see you replied. Sorry for chasing you like that, if that's how you feel. The reason for my rush, such as this preview, is because that your idea and the future direction of Geo Group as you described in your post have serious conflicts with mine. And our previous discussions always lead to nowhere. I'd like for us to reach some consensus before continue.

With your reply this time, I can see why you have such a problem with my emphasis on global CS. And I think you are right about one thing, we can't determine the type of coordinate system solely on the coordinate values. Just because one vertex is at (10,10,10), doesn't mean it should at the global (10,10,10). Your idea is to introduce a concept of local coordinate system, and change the meaning of (10,10,10) to refer to point in what every (nested) local coordinate the owner object belong. For a geo group type object, which is all about defining a coordinate system. this is totally fine. My real problem is that you want to extend this concept to every tool feature, meaning that what the tool feature claims must be in the same CS as the tools.

Is my understanding correct? If so, do you have any solution in mind to address my concern, that is, the tool feature from now on can only works with other objects inside the same CS? Or, you don't think that's an issue at all? Because, for one thing, it breaks existing FC geometry models. Or more precisely, models with shared children cannot exist in different App::Part from now on. Are there any benefits you can think of that outweighs the obvious problems of this restriction? Why the tools must care what CS its children belong? If your idea still requires all the children of the tool must be in the same CS anyway, why can't they just all be in the same global CS like they used to be? What really matters is what coordinate system the tool itself belong, is it not?

Oh, and if you need to look at the code, I'd suggest you read the Python demo code first. It has everything important in one place, much easier than going through C++ code spreading dozens of places.
Try Assembly3 (latest version 0.10.2) along with my custom build of FreeCAD at here.
And if you'd like to show your support, you can donate through patreon, liberapay, or paypal

ickby
Posts: 2906
Joined: Wed Oct 05, 2011 7:36 am

Re: Preview: Link, stage two, API groundwork

Post by ickby » Sun Jul 30, 2017 12:16 pm

It is not a concept for GeoFeatureGroup only, it is a general concept how we like to define the reference frame for absolutely every geo object in FreeCAD. Everything that has a placement (so everything derived from GeoFeature) needs a reference to which this placement is defined to. We have chosen to use a local approach, meaning that every object is referenced to the GeoFeatureGroup it is in. The only alternative would have been to reference it always to the global CS, which would have resulted in recalculating all GeoFeature objects when a Part would be moved.

So Every GeoFeature has a unambiguous location, it exists one time at one place only. That means every document object that uses geometric input from other objects need to know this unambiguous location to calculate the correct result. And this is only possible when the calculating object lives within the same GeoFeatureGroup than the objects it uses. Only than it uses the correct location information, and only then the calculated result is correct.

You are right, this means things like fusion are only allowed to live in the same GeoFeatureGroup as the objects it fuses. We are currently working on enforcing this. Coming to your concerns:

Toll and base in same Group: Yes, that will be enforced. Our solution to this is the instance object. If you want to use something in a different GeoFeatureGroup you simply make a instance of it and move that instance to the other Group. You have the object at two places, they are different. This is perfectly captured by the instance.

Breaking Models: Definitely not. GeoFeatureGroup or the whole concept has not been available before, we do not break anything. Not all constructions may be splittable into different GeoFeatureGroups, but that is a minor Issue in my opinion. All old workflows are still supportet, Everything can live in the global CS as they do now, just don't put it in a GeoFeatureGroup.

In general the restriction you noted is minor, compared to the obvious placement mess that would be created otherwise with your proposal.

User avatar
DeepSOIC
Posts: 6678
Joined: Fri Aug 29, 2014 12:45 am
Location: Saint-Petersburg, Russia

Re: Preview: Link, stage two, API groundwork

Post by DeepSOIC » Sun Jul 30, 2017 12:19 pm

realthunder wrote:
Sun Jul 30, 2017 3:05 am
DeepSOIC wrote:
Fri Jul 28, 2017 8:44 pm
... The question is, if I use "Part.Body.Face2", and Body happens to contain DocumentObject named Face2, what will be returned? Body.Shape.Face2, or Face2.Shape?
... So if there is an object named Face2, it will return that object instead of the subelement.

However, getSubObject has an option to either return Shape, or a tuple (Object,sublement,matrix,PyObj). ... If you ask for later,... the return tuple will have Object=Face2, subelement='', PyObj=Face2.Shape.
I was kind of nudging you to make a distinction by using a different separator in front of subelement, to be specific, and to allow to find out the type of shape returned from the link string alone. For example, "Part/Body.Face2" for Body.Shape.Face2, and "Part/Body/Face2" for Face2.Shape.

triplus
Posts: 8456
Joined: Mon Dec 12, 2011 4:45 pm

Re: Preview: Link, stage two, API groundwork

Post by triplus » Sun Jul 30, 2017 4:36 pm

Indeed local coordinate system. This is one of if not the biggest cornerstone of FreeCAD 0.17 development cycle. How should it all work and how should (existing) commands adapt to it. If this gets figured out and a high level of consensus is build around it. Issues like slow and limited workflow, command redundancy, bugs ... can and will for sure improve in next development cycles. As this are all nice to have but not crucial achievements in regards to FreeCAD 0.17.

Therefore in my opinion efforts like Link/Instance/Copy/Clone and Active Container/Coordinate System don't necessarily need to be a part of FreeCAD 0.17. They can happen early in FreeCAD 0.18 development cycle. But they should and need to put the pressure on the notion of local coordinate system in FreeCAD 0.17 development cycle. As there needs to be a certain level of comfort involved when adapting to it. And it don't have a feeling ATM we are there yet.

Post Reply