Merging of my Link branch

[fosselius]

Maybe we should start issues on the actual github linkstage3 page to have proper separation and followup. It will be messy to keep track of everything with 2 threads and several reporters.

Btw i think realthunder is close to a new release, things might have been left unfinished for now on the github project. What branch/commit did you test?

Have you tested the appimage?

I suspect that we will work with: (last commit 10h ago)
https://github.com/realthunder/FreeCAD/ ... /LinkMerge

[realthunder]

Ok I just gave the LinkStage3 branch a test spin, and read (rather quickly) the two docs (link features and core changes), and here are my impressions.

Finally seeing some response from the core developers. Thanks for the lengthy reply.

- The branch is pretty unstable. Several unit test fail, which is I think nothing serious, but the two existing models of mine I tried to open had serious problems, one got caught in infinite recompute loop and FreeCAD crashes quickly after, another after the apparently needed general recompute at file open (btw it is a design feature of FreeCAD to not need a recompute at startup) had half of its geometry screwed..

Please provide a sample file causing the crash. The general recompute thing is most likely because of the topological naming feature. It needs to generate names for legacy files. You can disable auto name generation on start up using parameter BaseApp/Preferences/Mod/Part/General/AutoElementMap. I leave at as True by default as this is an important feature to test.

I could easily make some more bugs happen by playing a couple of minutes with Draft and Arch tools. There are also some lags or freezes appearing, for example to switch between the data and view tabs, or for the statusbar to change on hovering an item in the tree (apparently because of the new highlight system)

If you are testing one of my pre-builts, then I have fixed a few performance problem, especially for python code. The performance lagging is again caused by new topological name mapping, because there is no easy way to guess whether the python code want the new names or not. There are internal caches to deal with this. But again, this could be something else. Anyway, I am about to release a new version of pre-built soon, hopefully by the end of this week.

- Running FreeCAD in command line mode apparently doesn't work anymore. I cannot open an existing document or run an import script (arch ifc importer). Something seem to have changed in the document py API

Right, haven't run the command line test for a long time. I'll do it soon.

- I am afraid this branch is impossible to merge as it is. It would likely screw a good portion of existing models

This is a chicken and egg problem, in order to make it work, I need more testers, more reports. FC can do so many different stuff, nobody uses all the feature.

- Why complicate Selection's SubName? Why not add other attribute such as Hierarchy? Seems to me easier to maintain backward compatibility. Many scripts out there might break with the proposed change... (It doesn't seem so at first sight, but I still need to test more)

That would require a lot more code, reinventing stuff like PropertyLinkSub and so on. The Selection API is modified such that its default behavior is the same as before, so existing code continues to work.

- Why the new python attach() method for document objects? Isn't it the same thing as __init__?

Because ViewProviderLink is designed to work with any object installed with LinkBaseExtension, so it needs a way to override the existing object's native view provider, using the App space python binding object, i.e. the App::FeaturePython's Proxy. Therefore, the core must allow user to create python binding object before creating the actually c++ object. The attatch() function is for the core to notify the python binding of object attaching. There is a code snippet shows how it works, specifically the way the object is created by the following call. Notice MyLink, the python binding, is created before the c++ object, so it can override the view provider using MyLink.getViewProviderName()

Code: Select all

link = obj.Document.addObject("App::FeaturePython",'link',MyLink(),None,True)

- Property status bit extension: what's the difference with the current setEditorMode()? Isn't it duplicating the same thing?

Editor mode uses two of the status bit, ReadOnly and Hidden to change the behavior of property editor. Other status bits have more general usage. For example Immutable controls the Python code read only access. However, the real reason I add this is for the user code to dynamically turn on Output and Transient status. The Tansient status allows the property to not be saved. For example, Draft Array does not need to save the compound if it is not fusing.

- recursive recomputing: I don't understand this... Isn't it what is already happening now?

Current recomputing recursively sort all the objects in dependency order, and recompute one by one with just one pass. My patch add a second pass in case any object is still touched after recomputing. This change to recompute is not essential for link to work. But I did rewrite the dependency object finding logic to account for the external linked objects. Besides, I've since added other alternatives solution for dependency inversion, like PropertyLinkHidden.

- if a document is partially loaded, can be edited by the user but shouldn't be edited and the user receives a warning and the changes are not saved, this seems an implementation flaw to me. The user should either be able to modify or not be able to modify at all.

That's more of a usability issue. Technically, the changes cannot be saved, because the document is incomplete. But maybe the user knows that and simply what to do some quick testing or whatever.

I understand it will require a huge work to isolate and extract things, but I really think we'll need to merge this step by step, feature by feature, starting with the big core changes one by one, without any of the more specific stuff. Otherwise I'm afraid there are high chances we'll simply throw all the stability we've been building in FreeCAD for years to the ground.

Yes, splitting requires a lot of repeated works, and prone to introducing new bugs. But even if I do this, some feature may still be too big, and there is still no guarantee of more testers. In fact, some new features will be more difficult to test when isolated, unlike now, all the new stuff can be tested in my Assembly3. The fact is, all of my changes are designed to be backward compatible, and for every release I do a merge with the upstream. So the first stage of testing is simple, and the same regardless of splitting or not. Just do your stuff as usual and see if there is anything broken. I can fix the problem quick if you can give me the problem file, otherwise you'll have to describe the problem in details.

To everybody else here, please test, and give feedback...

Yes, in the end of the day, this is the key to the problem here, more testers!

[DeepSOIC]

...FreeCAD crashes ... after the apparently needed general recompute at file open (btw it is a design feature of FreeCAD to not need a recompute at startup)

However, the real reason I add this is for the user code to dynamically turn on Output and Transient status. The Tansient status allows the property to not be saved. For example, Draft Array does not need to save the compound if it is not fusing.

BTW, I want this "recompute while loading instead of reading brep file" thing for Lattice pretty badly. When dealing with big arrays, recomputing an array is WAY faster than write/read the gigantic shape. I think the primary reason of slow read/write is that shape sharing is lost in the process. Lattice makes arrays using shallow copy, which makes FreeCAD/OCC store the actual geometry only once. Also true for triangulation. E.g. I can make a big array of screws with threads rather quickly, but saving/loading it takes forever.

Although changing OCC to store shape sharing is a better way in the long run.. and maybe it even does so already... I experimented with it quite long ago, things may have changed.

[yorik]

When dealing with big arrays, recomputing an array is WAY faster than write/read the gigantic shape. I think the primary reason of slow read/write is that shape sharing is lost in the process.

Interesting point..

Basically the best is to let the programmer of each feature decide. Hence the interest of the transient property bit...

Yes, in the end of the day, this is the key to the problem here, more testers!

Yes, let's start with that. I'll keep playing with your branch and try to isolate specific problems. But indeed it is very important that everybody who has some interest in seeing this enter FreeCAD do the same.

[Mark Szlazak]

End of this month is when Realthunder posted that he would have a release stable enough to merge. Merge it, fix things as they go and update with FreeCAD daily. The other worries are causing huge delays and I my bet is that they will not help. Fail fast, correct and repeat is an agile approach.

[sgrogan]

End of this month is when Realthunder posted that he would have a release stable enough to merge. Merge it, fix things as they go and update with FreeCAD daily. The other worries are causing huge delays and I my bet is that they will not help. Fail fast, correct and repeat is an agile approach.

IMHO, I disagree. We basically did this for PDN and it took 2 yrs to make a release. PDN had a small team, on this realthunder is an incredible 1 man show.

@realthunder's development is model is impeccable. He re-bases on master at every release and provides builds for all 3 OS's. So really realthunder's releases are not much different than if the code was in master. We could post the builds in a more official place like the FreeCAD/releases page, maybe some of us packagers could try to help offloading some of the build duties from @realthunder. Remember that there are 2 things at play here; LinkStage3 and asm3. LinkStage3 has to be merged in master, and asm3 needs to be an add-on due to the solve-space licence. @realthunder has designed asm3 to use different solvers so this is an intermediate problem.

We need a strategy to get more testers and not alienate regular users. Maybe we could have a LinkStage3 branch on FreeCAD's GitHub Repo, there were some when we were on SourceForge, but it seems a maintenance nightmare.

Please don't take anything here as negative, it's not. I think this will be a tough nut to crack but we will get there.

[easyw-fc]

We could post the builds in a more official place like the FreeCAD/releases page
....
We need a strategy to get more testers and not alienate regular users. Maybe we could have a LinkStage3 branch on FreeCAD's GitHub Repo...

+1
more users will test the @realthunder's release, more feedback will be available and more option will be available to see how his work can/could impact the different official FC workbenches...

I just want to say that, imo, @realthunder work is great not only because of asm3, but even more for introducing Links, improving STEP import/export options and highly improve OpenGL performance...

These three features are a very good reason theirself to try to find a way to merge his code and work in FC main repo...
Asm3 will follow easily, because it is based on LinkStage3 core code...

I hope it will be more interaction among the FC developers to find a way to go in a converging direction

Maurice

[looo]

I already tested some basic stuff of assembly3-module with python3 and this worked. The modules I am caring about are also working. Loading a glider takes much more time with the link-branch (about 5x). But I guess this is more related to how I implemented the document-loading, so maybe time to get a better understanding of the document loading mechanism in FreeCAD ...

on my machine loading the wikihouse from: https://github.com/uncreatednet/wikilab-ufabc
needs 41.5 sec for python2.7 / qt4 / master-branch
needs 60.7 sec for python3.6 / qt5.6 / link-branch
needs 39.2 sec for python3.6 / qt5.6 / master-branch

End of this month is when Realthunder posted that he would have a release stable enough to merge. Merge it, fix things as they go and update with FreeCAD daily. The other worries are causing huge delays and I my bet is that they will not help. Fail fast, correct and repeat is an agile approach.

I guess reviewing 50000 lines is a bit more difficult... In my eyes splitting the diff in smaller patches is the best way to proceed.

We need a strategy to get more testers and not alienate regular users. Maybe we could have a LinkStage3 branch on FreeCAD's GitHub Repo, there were some when we were on SourceForge, but it seems a maintenance nightmare.

I don't think a more official branch will change anything. Maybe we could change the ppa to the link-branch for some days? I guess there will be a lot of topics why xy is broken .

[fosselius]

As realthunder is rebasing and keeping up to date with the master branch, everything that is in the freecad repo is already included in realthunders branch.

Splitting it up to smaller merges would be a terrible waste of time. We can already test, find and solve issues with the complete package of changes and work on it to make it stable enough for all workbenches to merge into master.

We could possibly "fork" it back into the master repo as FreeCAD/next? Alpha? Bleedingedge? Or just FreeCAD/LinkStage3 To give other devs access and control?

Maybe its more efficient if everybody just creates pull requests for realthunder for now?

[DeepSOIC]

Maybe its more efficient if everybody just creates pull requests for realthunder for now?

I already wanted to do that once. Planned a small but important Py api expansion. However, Abdullah managed to suck me into helping with sketcher section tool.