STEP file read might be crazy long ...

Here's the place for discussion related to coding in FreeCAD, C++ or Python. Design, interfaces and structures.
User avatar
vejmarie
Posts: 680
Joined: Mon Jan 04, 2016 4:52 pm
Location: Somewhere between France, USA and Taiwan
Contact:

Re: STEP file read might be crazy long ...

Postby vejmarie » Mon Jan 30, 2017 3:07 pm

easyw-fc wrote:
sgrogan wrote:Win Test build and sample file: https://github.com/sgrogan/FreeCAD/releases/tag/STEP
thx a lot!
I will be able to test it this evening...
It would be nice to test it comparing also with CAD-Assistant for Windows of OCC
https://www.opencascade.com/content/cad-assistant
It is a CAD viewer also for STEP models and it is based on OCC and Qt, so a comparison may be interesting, to see if FC with MT is using all the power of OCC
vejmarie wrote: ...
- 2 - Selection of Compound which contains a huge number of elements which is crazy slow and make navigation painfull if these Compound are active.
...
vejmarie
CAD Assistance is available also for OSX, so it may worth to make a comparison with your new MT branch to CAD-Assistant
In Windows, after a very fast test, it seems to be much more responsive than FC in selecting and displaying elements
Thx as always
Maurice
Hi Maurice,

Yes selecting and displaying elements in FC is a big mess currently. This is where I will try to focus after, as I have to manipulate STEP files which contains thousands of elements, and currently the selection algorithm is so slow on Compound that even if my tree hierarchy works it will be "useless" if we can't accelerate the rendering (this is explaining why I am currently "hiding" Compound from the rendering window as to accelerate graphical selection and tree manipulation). The code is quite complex on this part, but it works fine ;). Let's try to understand what's under the hood and what's going on.

I am starting to think that we need an HOW-TO section for developpers. I am new to FreeCAD and understanding all the connection between each part of the code is taking me a crazy amount of time.

vejmarie
User avatar
Kunda1
Posts: 8759
Joined: Thu Jan 05, 2017 9:03 pm

Re: STEP file read might be crazy long ...

Postby Kunda1 » Mon Jan 30, 2017 3:27 pm

vejmarie wrote:I am starting to think that we need an HOW-TO section for developpers. I am new to FreeCAD and understanding all the connection between each part of the code is taking me a crazy amount of time.
@vejmarie what would you like to see in this Dev HOWTO ?
Want to contribute back to FC? Checkout:
#lowhangingfruit | Use the Source, Luke. | How to Help FreeCAD | How to report FC bugs and features
User avatar
vejmarie
Posts: 680
Joined: Mon Jan 04, 2016 4:52 pm
Location: Somewhere between France, USA and Taiwan
Contact:

Re: STEP file read might be crazy long ...

Postby vejmarie » Mon Jan 30, 2017 5:33 pm

Stupid things like :
- How to hide/show a component into the tree by using C++ code
- Global code structure
- Signaling, what is called and when. Class interaction
- and many more things ... like basic OpenCascade example etc
User avatar
easyw-fc
Posts: 2915
Joined: Thu Jul 09, 2015 9:34 am

Re: STEP file read might be crazy long ...

Postby easyw-fc » Mon Jan 30, 2017 10:17 pm

sgrogan wrote:Win Test build and sample file: https://github.com/sgrogan/FreeCAD/releases/tag/STEP
vejmarie wrote:...
here some of my test results:
FC 9802 64b
loading time = 1236.62700009s
displaying time = 92.0999999046s
tot. 1328s

FC vejmarie STEP TBBx64
loading time = 971.539000034s
after a lot of errors like the following
PropertyContainer::Save: Unknown C++ exception thrown. Try to continue...
displaying time = 464.898000002s
tot. 1436s
(loading time is faster, displaying time is slower .... probably the total time is afflicted by exception errors when displaying the model)

CAD Assistant
loading 37.426s
prepared in 196.581
presentation in 81.9969s
tot. 316s

DesignSparks Mechanical
loading and displaying time
tot. 330s

Notes: both CAD-A and DSM are much more responsive compared to FC when rotating and selecting objects in the loaded STEP file ...

OS: Windows 10 Pro
Word size of OS: 64-bit
Word size of FreeCAD: 64-bit

on a
GenuineIntel Intel(R) Core(TM) i5 CPU 750 @ 2.67GHz 4 cores
and the script I used to calculate loading and displaying time

Code: Select all

#!/usr/bin/python
# -*- coding: utf-8 -*-
#****************************************************************************

import time

def say(msg):
    FreeCAD.Console.PrintMessage(msg)
    FreeCAD.Console.PrintMessage('\n')

say("loading... ")
t = time.time()
import ImportGui
ImportGui.open(u"C:\\Temp\\steptest\\00_server-tla_25ou-barreleye_asm.stp")
#Gui.SendMsgToActiveView("ViewFit")
timeP = time.time() - t
say("loading time = "+str(timeP) + "s")
Gui.SendMsgToActiveView("ViewFit")
timeD = time.time() - t - timeP
say("displaying time = "+str(timeD) + "s")
peterl94
Posts: 1001
Joined: Thu May 23, 2013 7:31 pm
Location: United States

Re: STEP file read might be crazy long ...

Postby peterl94 » Mon Jan 30, 2017 10:27 pm

vejmarie wrote:I am starting to think that we need an HOW-TO section for developers
Sorry for an off-topic and likely irrelevant post, but I happened to read your post and it reminded me of this page I have bookmarked that I intend to explore when I have time to get into FreeCAD development. http://freecadweb.org/wiki/index.php?ti ... .27s_guide
User avatar
vejmarie
Posts: 680
Joined: Mon Jan 04, 2016 4:52 pm
Location: Somewhere between France, USA and Taiwan
Contact:

Re: STEP file read might be crazy long ...

Postby vejmarie » Tue Jan 31, 2017 8:19 am

easyw-fc wrote:
sgrogan wrote:Win Test build and sample file: https://github.com/sgrogan/FreeCAD/releases/tag/STEP
vejmarie wrote:...
here some of my test results:
FC 9802 64b
loading time = 1236.62700009s
displaying time = 92.0999999046s
tot. 1328s

FC vejmarie STEP TBBx64
loading time = 971.539000034s
after a lot of errors like the following
PropertyContainer::Save: Unknown C++ exception thrown. Try to continue...
displaying time = 464.898000002s
tot. 1436s
(loading time is faster, displaying time is slower .... probably the total time is afflicted by exception errors when displaying the model)

CAD Assistant
loading 37.426s
prepared in 196.581
presentation in 81.9969s
tot. 316s

DesignSparks Mechanical
loading and displaying time
tot. 330s

Notes: both CAD-A and DSM are much more responsive compared to FC when rotating and selecting objects in the loaded STEP file ...
Thanks for the test report ;) which do correspond to what I see. By the way my code is focused on the loading time. The displaying time is probably the issue I am observing which is when there is a lot of elements standing into a Compound the renderer is super slow. As my algorithm is using Compound to build the STEP hierarchy tree we are ending up into this issue. Which commit is your build using ? Probably not the latest one (sgrogan is going to "kick me"), because wwmayer did integrated yesterday night my latest commit that do de-activate Compound display and purely render end object contained into Compound. So I think that with this latest patch you will get something like loading time around 971s, Displaying around 90s. That is not a massive improvement compared to original code, but still 30% and a STEP tree.

My current feeling (and this is a pure feeling) is that Compound are recomputed each time they are selected, and I think this is a bad idea, because this is a super long task if they do contain a lot of Shapes. They should be recomputed only when a modification happens within the underlying shapes. This is just my feeling which need to be proved or the issue might be coming from somehwhere else.

I don't know how CAD Assistant and DesignSparks address the decoding of STEP, but I make the bet that they are not relying on the super complex infrastructure that we do have. The STEP code reader in Open Cascade is a very old piece of code, which is not optimal and complex. I think we are ending up into the worse part of what C++ might be. I will continue the effort.

But I loved to see the Compound rendering/selection performance issue fixed as this is standing into FreeCAD code, and this is a pain. One of my colleague who design things with FreeCAD is feeling this slowness while using the software under linux and we ends up in situation where it is unusable which is bad. This is where I am focused now. I will then address the Compound color creation issue. I have checked the code, and in that case I am sure of something, the Part::Compound creation code doesn't copy the Color of the sub shape within the Compound which explain why every Compound element are grey even if subshapes do contains colors.

We are progressing, we are progressing, let's keep moving
Jee-Bee
Posts: 2145
Joined: Tue Jun 16, 2015 10:32 am
Location: Netherlands

Re: STEP file read might be crazy long ...

Postby Jee-Bee » Tue Jan 31, 2017 9:02 am

vejmarie wrote:The STEP code reader in Open Cascade is a very old piece of code, which is not optimal and complex. I think we are ending up into the worse part of what C++ might be. I will continue the effort.
So the biggest improvement would be that OCC upgrade their step exporter!!
User avatar
vejmarie
Posts: 680
Joined: Mon Jan 04, 2016 4:52 pm
Location: Somewhere between France, USA and Taiwan
Contact:

Re: STEP file read might be crazy long ...

Postby vejmarie » Tue Jan 31, 2017 9:11 am

Jee-Bee wrote:
vejmarie wrote:The STEP code reader in Open Cascade is a very old piece of code, which is not optimal and complex. I think we are ending up into the worse part of what C++ might be. I will continue the effort.
So the biggest improvement would be that OCC upgrade their step exporter!!
I think the biggest improvement might be that we help them to improve that part. We also need to improve our rendering capability, because I think we can reach something like 500s on bareleye model with the current infrastructure. This still will be 40-50% slower speed than competitive reader, but this might be acceptable. I have some room of improvement within my code, but need to get access to more powerful system than my macbook machine which doesn't have enough core to understand what's going on in the parallel code.

We definitly need to run FreeCAD into tools like Instruments on MacOS to track performance issues, and just looking at the call stack afraid me in some cases. But another time this is fine, that is a great project with crazy contributors and everything is there to make it work greatly. STEP reader is mostly legacy work, and Open Cascade is probably not use that much for such purpose. Let's pursue our work on that and see what we could do to improve it. By the way this first step see some 30% improvement. If we improve everything by 30% every 6 months, that might be a really great run rate ;)
ickby
Posts: 2985
Joined: Wed Oct 05, 2011 7:36 am

Re: STEP file read might be crazy long ...

Postby ickby » Tue Jan 31, 2017 9:20 am

Is there a special reason you use compound instead of the new Part Object, which is exactly intended for such purposes? (Or a special GeoFeatureGroup if no origin is wanted)? As a Compound always douplicates the child shapes this seems very resource intense, both calculation cycles and memory wise. The Part object only shifts objects and hence is very lightweight.

Disclaimer: Currently it is not possible to have a object in multiple places, so references won't work (you would still need to duplicate the shape here). That will be possible with instances in the near future. Also some things regarding Parts are not yet super solid. But would be a good proof of concept if step import would create Parts already.
User avatar
vejmarie
Posts: 680
Joined: Mon Jan 04, 2016 4:52 pm
Location: Somewhere between France, USA and Taiwan
Contact:

Re: STEP file read might be crazy long ...

Postby vejmarie » Tue Jan 31, 2017 10:37 am

ickby wrote:Is there a special reason you use compound instead of the new Part Object, which is exactly intended for such purposes? (Or a special GeoFeatureGroup if no origin is wanted)? As a Compound always douplicates the child shapes this seems very resource intense, both calculation cycles and memory wise. The Part object only shifts objects and hence is very lightweight.

Disclaimer: Currently it is not possible to have a object in multiple places, so references won't work (you would still need to duplicate the shape here). That will be possible with instances in the near future. Also some things regarding Parts are not yet super solid. But would be a good proof of concept if step import would create Parts already.
Hi ickby, except my lack of creativity and ignorance on the new Part Object no specific reasons ;). Let me try to have a look to it and see if I can adapt the code, it shouldn't be that hard.