Inefficient handling of _GroupTouched

Here's the place for discussion related to coding in FreeCAD, C++ or Python. Design, interfaces and structures.
Forum rules
Be nice to others! Respect the FreeCAD code of conduct!
Post Reply
matthijskooijman
Posts: 72
Joined: Thu Mar 25, 2021 10:59 am

Inefficient handling of _GroupTouched

Post by matthijskooijman »

I was reading up on the visibility handling and noticed the _GroupTouched property, and I think there might be a significant and unnecessary overhead there. Here's what I found:
  • App::GroupExtension adds a "_GroupTouched" transient property, which is always false.
  • App::GroupExtension::extensionOnChanged registers a changed handler for each child object inside the group
  • App::GroupExtension::slotChildChanged is that handler and is called for every change to a child object. When a child's visibility changes, the _GroupTouched property of the group is touched. This behavior was introduced in this commit.
  • I presume that touching this property, causes the group itself to be touched and marked for recompute?
  • Some classes that use GroupExtension (PartDesign body and DocumentObjectGroup), mark the _GroupTouched property as "Output". Looking at the git commits because they do not want to recompute when a child visibility changes, so I'm assuming that touching an output property does not cause the object to become touched/recomputed, so this effectively suppresses these recomputes for PartDesign::Body and DocumentObjectGroup? Commits here and here, discussion for the latter here
IIUC this correctly, this means that all groups register changed handlers on all their children and do touching of _GroupTouched, even when (for Body and DOG), this produces no effect at all? Wouldn't it be a lot simpler (and more efficient) to add a "touchedByChildrenVisibility" boolean in GroupExtension or so and letting Body and DOG set that to false, and when it is false, completely skip the changed handler registration? Or am I misunderstanding something and would that produce different behavior?

Also, I wonder why bother with this _GroupTouched property at all? Why not just touch the group object itself directly, rather than this dummy property? It doesn't seem that there is anyone listening for this particular property either? It also seems that "GroupExtension::extensionExecute" also touches the _GroupTouched property, though I'm not quite sure why. I'm not really familiar with how this touching works, so maybe I'm missing something obvious here.

I also found some related discussion here (haven't read all of it yet), which seems to have resulted in a PR by Realthunder fixing some view-related problems that seem to very much touch upon this area, but looking at the changes, do not fix this particular issue. Still, maybe RT has some insights here?
realthunder wrote: ping
matthijskooijman
Posts: 72
Joined: Thu Mar 25, 2021 10:59 am

Re: Inefficient handling of _GroupTouched

Post by matthijskooijman »

I noticed another weird handling related to _GroupTouched. Consider a Group inside a BuildingPart:
  • With the inner group selected, press space, which calls ViewProvider::show() on the inner group, which sets Visible=False
  • This triggers the GroupExtension on the outer BuildingPart, which touches its _GroupTouched property
  • This triggers Gui::ViewProvider::update (via Gui::Document::slotChangedObject), which does something peculiar: it *hides* the updated object, then calls extensionUpdateData on all its extensions, and then *shows* the object (the outer BuildingPart) again.
  • This then causes ViewProviderGroupExtension::extensionShow, as it should, to set the Visibility of all its invisible children to true
  • Then, because the inner group becomes visible again the above sequence actually *repeats* once more, but because ViewProviderGroupExtension::extensionShow has a recursion guard, this does not become an infinite recursion.
  • The end result, is that the inner group effectively stays visible.
The above sequence was discovered using some gdb breakpoints, so I'm reasonably sure it is accurate.

Interestingly enough I could only reproduce this behavior for a newly created buildingpart/group in an isolated testfile, after saving and restoring that file, I ended up with a BuildingPart that was marked as invisible (grayed out) even though it had Visibility=True, and gdb showed that toggling visibility actually tried to *show* it (isShow() apparently returned false, because pcModeSwitch->whichChild was -1, but because viewOverrideMode was 0x48ffbcb6 (seems corrupt AFAICT, should be a low number), setModeSwitch would refuse to change whichChild and the BuildingPart can no longer be made visible... Maybe a different bug, maybe related, dunno.


Looking at the above, I wonder (again) why this _GroupTouched stuff is really needed. But also, why does ViewProvider::update actuall do this trick of hiding, applying an update and then showing? The comment claims to speed up updates, but at first glance, this seems like a rather fragile and also ineffective way: Instead of doing a potentially slow redraw (I guess that's what's being prevented?), it now ends up triggering a whole bunch of stuff because of the hide and show. And if a bunch of properties are updated in a single transaction, you'd end up doing this a bunch of times (and I guess everytime you call show, a redraw is triggered anyway?). I'm not too familiar with the code, but if the problem is slow redraws during updates, wouldn't it be better to globally implemented delayed redraws anyway (i.e. mark things as dirty while updating data, then afterwards redraw all dirty objects). Maybe there is something else going on, though.

Looking further into the code, I've noticed that ViewProviderDocumentObject::updateView also does the same trick, but it sets the User1 bit on the Visibility property, to prevent further syncing of the visibility property. Additionally, ViewProviderDocumentObject::update also sets this bit, so it is in effect when ViewProviderGroupExtension::extensionShow is called. And looking at ViewProviderGroupExtension::extensionHide, that actually checks for this bit and does not propagate visibility when it is set, so maybe this is how it is intended to work (though again, this looks fragile at first glance: What if some property update actually triggers a genuine visibility update, maybe in some user Python code, which would now, I think, not be propagated to children because it is indistinguishable from the temporary changes).

The User1 check in ViewProviderGroupExtension::extensionHide was introduced in this commit by Realthunder. At first glance, it looks like an oversight that extensionHide was modified but extensionShow was not, and an obvious and quick fix would be to add the same check to extensionShow as well (haven't tried it yet, though).

Looking at Realthunder's PR 2759, I see that that also makes extensive changes to this area, but from a first glance there, the code that toggles child visibilities was moved and refactored in this commit, which actually drops the User1 check. There is a new _checkParentGroup guard that prevents visibility propagation, but from the looks of it, that is used for a different purpose. And looking at the other code, the _GroupTouched and ViewProvider::update behavior that causes this problem is still there. The PR does modify the visibility handling to save the old visibility state of children, so maybe that could mask this problem (it would look fixed, but do a lot of unneeded churning on visibilities in the background).

matthijskooijman wrote: Wed May 19, 2021 4:45 pm Also, I wonder why bother with this _GroupTouched property at all? Why not just touch the group object itself directly, rather than this dummy property?
I think I realized the reason for this: When touching _GroupTouched which is marked as Output, then the group itself is *not* touched, but anyone listening for changes in the group is notified, so this can e.g. touch other objects that reference the group when needed (without those objects having to listen to, or even be aware of, group members directly). I guess using a dummy property in this way is still a bit confusing, but it's probably a pragmatic way to handle this. I do wonder if there would be a cleaner and/or more semantic way to handle this, maybe a nicer name for the property could already help.
Last edited by matthijskooijman on Wed May 19, 2021 11:44 pm, edited 1 time in total.
matthijskooijman
Posts: 72
Joined: Thu Mar 25, 2021 10:59 am

Re: Inefficient handling of _GroupTouched

Post by matthijskooijman »

matthijskooijman wrote: Wed May 19, 2021 8:45 pm Interestingly enough I could only reproduce this behavior for a newly created buildingpart/group in an isolated testfile, after saving and restoring that file, I ended up with a BuildingPart that was marked as invisible (grayed out) even though it had Visibility=True, and gdb showed that toggling visibility actually tried to *show* it (isShow() apparently returned false, because pcModeSwitch->whichChild was -1, but because viewOverrideMode was 0x48ffbcb6 (seems corrupt AFAICT, should be a low number), setModeSwitch would refuse to change whichChild and the BuildingPart can no longer be made visible... Maybe a different bug, maybe related, dunno.
Turns out that this was indeed a completely unrelated bug. The weird values for viewOverrideMode were because gdb didn't read function arguments (this pointer) correctly until I stepped on line into the function, so the viewOverrideMode variable turned out to be innocent :-) See https://forum.freecadweb.org/viewtopic.php?f=10&t=58769 for details about what happened here.
matthijskooijman wrote: Wed May 19, 2021 8:45 pm The User1 check in ViewProviderGroupExtension::extensionHide was introduced in this commit by Realthunder. At first glance, it looks like an oversight that extensionHide was modified but extensionShow was not, and an obvious and quick fix would be to add the same check to extensionShow as well (haven't tried it yet, though).
I just tried applying this small fix and it indeed seems to fix this particular problem. I'm not yet sure if there are additional side effects, though...
realthunder
Veteran
Posts: 2190
Joined: Tue Jan 03, 2017 10:55 am

Re: Inefficient handling of _GroupTouched

Post by realthunder »

The reason of having _GroupTouched is to solve the missing group content change notification, especially for nested group. Because normally, if a group member changes, the group itself has no property changes. So the group, and higher parent groups won't receive notification in its onChanged() function. The _GroupTouched property solve this problem.

The Part::Feature::getTopoShape() function relies on this to manage internal compound shape caches for the group. With the current upstream, making a compound normally isn't that expensive, but still takes time if calling from a loop. It will become noticable if there is any scaling involved with Link, which can link to any object, including a group. With my topo naming code, caching becomes even more needed. BTW, Link also faces similar notification problem, and has _LinkTouched property for that purpose.

The PR I submitted is to provide a way for user code to disable this _GroupTouched notification mechanism, through the new ExportMode property. For example, PartDesign::Body does not need it, because body manage its own shape, which only follows the tip feature. It will modify its Shape property when the tip feature changes.

The behavior of ViewProvider hiding before update exists before the earliest history log in FreeCAD github repo. In my PR, I have changed GroupExtension to monitor DocumentObject.Visibility changes of its children, instead of their view provider Visibility. The DocumentObject.Visibility and ViewProviderDocumentObject.Visibility will normally sync with each other on user change, but will not in case of hiding before update. See ViewProviderDocumentObject::updateView().

There are other changes here and there regarding group handling accumulated in my own branch. I'll update my PR when I have time.
Try Assembly3 with my custom build of FreeCAD at here.
And if you'd like to show your support, you can donate through patreon, liberapay, or paypal
matthijskooijman
Posts: 72
Joined: Thu Mar 25, 2021 10:59 am

Re: Inefficient handling of _GroupTouched

Post by matthijskooijman »

realthunder wrote: Thu May 20, 2021 5:39 am The reason of having _GroupTouched is to solve the missing group content change notification, especially for nested group. Because normally, if a group member changes, the group itself has no property changes. So the group, and higher parent groups won't receive notification in its onChanged() function. The _GroupTouched property solve this problem.
Right, that makes sense, I guess. Though the name is still a bit confusing, since _GroupTouched only works in *visibility* changes in the group contents, not any other changes, right (unlike _LinkTouched, which seems to work on all (non-output) properties)?
realthunder wrote: Thu May 20, 2021 5:39 am The PR I submitted is to provide a way for user code to disable this _GroupTouched notification mechanism, through the new ExportMode property. For example, PartDesign::Body does not need it, because body manage its own shape, which only follows the tip feature. It will modify its Shape property when the tip feature changes.
Yeah, I like the idea behind the ExportMode property, since it seems like a nice and declarative way to specify the behavior of a group. I'm not entirely sure if this should be a stored property (just using it for object-type-specific code to indicate what they want to happen seems like an obvious choice, and specific types could maybe make it a stored, non-hidden property if they want to expose the choice to the user, at the expense of more compatibility to maintain), but maybe I'm missing the obvious usecases here. As for the code that implements this, I had a good look through its commit, but there's *so many* changes in there, that I can't really comment on any of it (other than that it looks like a lot of extra complexity and special casing is still being introduced, which I'm not sure is a good thing).

As for the performance issue this thread started about: It seems that it is not really fixed in that PR (it still registers handlers for visibility changes for *all* children. However, since the change handler now does more things (some of which I don't understand, some of which are to keep track of children's visibility states to restore them when showing the group itself), the problem might have been gone away (the overhead is still there, but it's no longer unneccessary). Also, because it now only listens to Visibility changes, the overhead in terms of signal handlers being *called* is lower, even if the amount of handlers *registered* is the same.
realthunder wrote: Thu May 20, 2021 5:39 am The behavior of ViewProvider hiding before update exists before the earliest history log in FreeCAD github repo. In my PR, I have changed GroupExtension to monitor DocumentObject.Visibility changes of its children, instead of their view provider Visibility. The DocumentObject.Visibility and ViewProviderDocumentObject.Visibility will normally sync with each other on user change, but will not in case of hiding before update. See ViewProviderDocumentObject::updateView().
Right, I hadn't realized before that the object also has a Visibility, but I see now it is a hidden property. I did recently found some code that synced this property and was confused by it. Just when I thought I started to understand things a bit, there's another layer of complexity... :-( Oh well, let's see if I can figure this out.

Thanks for your clarifications in any case.
Post Reply