STEP file read might be crazy long ...

Here's the place for discussion related to coding in FreeCAD, C++ or Python. Design, interfaces and structures.
User avatar
vejmarie
Posts: 680
Joined: Mon Jan 04, 2016 4:52 pm
Location: Somewhere between France, USA and Taiwan
Contact:

STEP file read might be crazy long ...

Postby vejmarie » Wed Aug 10, 2016 9:04 pm

Hi,

I am trying to open the STEP file within this Open Compute archive http://files.opencompute.org/oc/public. ... 7&download which is a Rackspace server. My Macbook is annoucing me 20h to read the file. So I will wait for the process to end up (I am not running out of memory ;), so no swap)

Some ProE info from the export

Pro/ENGINEER INTERFACE EXPORT LOG FILE

STEP output for 装配 00_SERVER-TLA_25OU-BARRELEYE.



Option values used for this export.

step_export_format AP203_IS
export_3d_force_default_naming NO
intf3d_out_extend_surface YES
intf3d_out_surface_deviation DEFAULT
intf_out_as_bezier NO
intf3d_out_force_surf_normals NO
intf_out_max_bspl_degree 16


Process began at: Fri Jun 24 14:45:19 2016

Process finished at: Fri Jun 24 14:47:28 2016

Total process time: 00:02:09.


MODEL: 00_SERVER-TLA_25OU-BARRELEYE_ASM
OUTPUT FILE: 00_server-tla_25ou-barreleye_asm.stp

ADVANCED_BREP_SHAPE_REPRESENTATION 285
ADVANCED_FACE 109373
APPLICATION_CONTEXT 1
APPLICATION_PROTOCOL_DEFINITION 1
APPROVAL 1
APPROVAL_DATE_TIME 1
APPROVAL_PERSON_ORGANIZATION 1
APPROVAL_ROLE 1
APPROVAL_STATUS 1
AXIS1_PLACEMENT 4
AXIS2_PLACEMENT_3D 176573
BOUNDED_SURFACE 1755
BREP_WITH_VOIDS 5
B_SPLINE_CURVE_WITH_KNOTS 13710
B_SPLINE_SURFACE 1755
B_SPLINE_SURFACE_WITH_KNOTS 2926
CALENDAR_DATE 1
CARTESIAN_POINT 893713
CC_DESIGN_APPROVAL 1
CC_DESIGN_DATE_AND_TIME_ASSIGNMENT 2
CC_DESIGN_PERSON_AND_ORGANIZATION_ASSIGNMENT 4
CC_DESIGN_SECURITY_CLASSIFICATION 1
CIRCLE 68205
CLOSED_SHELL 362
CONICAL_SURFACE 3436
CONTEXT_DEPENDENT_SHAPE_REPRESENTATION 1246
CONVERSION_BASED_UNIT 416
COORDINATED_UNIVERSAL_TIME_OFFSET 1
CYLINDRICAL_SURFACE 29483
DATE_AND_TIME 1
DATE_TIME_ROLE 2
DESIGN_CONTEXT 201
DIMENSIONAL_EXPONENTS 416
DIRECTION 569145
EDGE_CURVE 296202
EDGE_LOOP 121713
FACE_BOUND 12340
FACE_OUTER_BOUND 109373
FILE_DESCRIPTION 1
FILE_NAME 1
FILE_SCHEMA 1
GEOMETRIC_REPRESENTATION_CONTEXT 402
GEOMETRIC_REPRESENTATION_ITEM 1755
GLOBAL_UNCERTAINTY_ASSIGNED_CONTEXT 402
GLOBAL_UNIT_ASSIGNED_CONTEXT 402
ITEM_DEFINED_TRANSFORMATION 1246
LENGTH_MEASURE_WITH_UNIT 14
LENGTH_UNIT 416
LINE 215143
LOCAL_TIME 1
MANIFOLD_SOLID_BREP 343
MECHANICAL_CONTEXT 402
NAMED_UNIT 1622
NEXT_ASSEMBLY_USAGE_OCCURRENCE 1246
ORGANIZATION 1
ORIENTED_CLOSED_SHELL 14
ORIENTED_EDGE 592404
PERSON 1
PERSON_AND_ORGANIZATION 1
PERSON_AND_ORGANIZATION_ROLE 4
PLANE 68343
PLANE_ANGLE_MEASURE_WITH_UNIT 402
PLANE_ANGLE_UNIT 804
PRODUCT 402
PRODUCT_CATEGORY 2
PRODUCT_CATEGORY_RELATIONSHIP 2
PRODUCT_DEFINITION 402
PRODUCT_DEFINITION_FORMATION_WITH_SPECIFIED_SOURCE 402
PRODUCT_DEFINITION_SHAPE 1648
PRODUCT_RELATED_PRODUCT_CATEGORY 2
RATIONAL_B_SPLINE_SURFACE 1755
REPRESENTATION_CONTEXT 402
REPRESENTATION_ITEM 1755
REPRESENTATION_RELATIONSHIP 1246
REPRESENTATION_RELATIONSHIP_WITH_TRANSFORMATION 1246
SECURITY_CLASSIFICATION 1
SECURITY_CLASSIFICATION_LEVEL 1
SHAPE_DEFINITION_REPRESENTATION 402
SHAPE_REPRESENTATION 117
SHAPE_REPRESENTATION_RELATIONSHIP 1246
SI_UNIT 1206
SOLID_ANGLE_UNIT 402
SPHERICAL_SURFACE 873
SURFACE 1755
SURFACE_OF_LINEAR_EXTRUSION 852
SURFACE_OF_REVOLUTION 4
TOROIDAL_SURFACE 3456
UNCERTAINTY_MEASURE_WITH_UNIT 402
VECTOR 215995
VERTEX_POINT 189229

TOTAL 3723233
I know that STEP import is coming from OCCT probably and we can't do anything, but we might perhaps help it to go faster as 20h is quite long to read a file which has been created in 2 minutes
jmaustpc
Posts: 10377
Joined: Tue Jul 26, 2011 6:28 am
Location: Australia

Re: STEP file read might be crazy long ...

Postby jmaustpc » Wed Aug 10, 2016 10:53 pm

vejmarie wrote:I am trying to open the STEP file within this Open Compute
please post your FreeCAD version data, which also includes your OCC or OCE version.
jmaustpc
Posts: 10377
Joined: Tue Jul 26, 2011 6:28 am
Location: Australia

Re: STEP file read might be crazy long ...

Postby jmaustpc » Wed Aug 10, 2016 10:54 pm

that file is 219Mb and that is as a zip file.
jmaustpc
Posts: 10377
Joined: Tue Jul 26, 2011 6:28 am
Location: Australia

Re: STEP file read might be crazy long ...

Postby jmaustpc » Wed Aug 10, 2016 11:20 pm

vejmarie wrote:I am trying to open the STEP file within this Open Compute archive http://files.opencompute.org/oc/public. ... 7&download which is a Rackspace server.
Have you perhaps linked to the wrong file? Your link downloads a huge zip file containing all sorts of things including a BIOS and PDF files etc. I did find a stp file in a sub directory which when unzipped was also 199.9MB and I tried importing it in a very old FreeCAD 0.14 and after a few minutes I gave up.
User avatar
vejmarie
Posts: 680
Joined: Mon Jan 04, 2016 4:52 pm
Location: Somewhere between France, USA and Taiwan
Contact:

Re: STEP file read might be crazy long ...

Postby vejmarie » Wed Aug 10, 2016 11:52 pm

jmaustpc wrote:
vejmarie wrote:I am trying to open the STEP file within this Open Compute archive http://files.opencompute.org/oc/public. ... 7&download which is a Rackspace server.
Have you perhaps linked to the wrong file? Your link downloads a huge zip file containing all sorts of things including a BIOS and PDF files etc. I did find a stp file in a sub directory which when unzipped was also 199.9MB and I tried importing it in a very old FreeCAD 0.14 and after a few minutes I gave up.
The step you found is the right one ;) it is in ME directory
Sorry for the archive it is huge I know and does contains all the design file from the project. I didn't found yet an easy way to post it somewhere (I am using a roaming data connexion currently). So reading the file takes long time because it is big, but I think we might be able to improve this time, or it is a good exercise to try to open it.

I am not fully up to date, but this is the current version I am using on my Mac.

OS: Mac OS X
Word size of OS: 64-bit
Word size of FreeCAD: 64-bit
Version: 0.17.7990 (Git)
Build type: Unknown
Branch: master
Hash: da70ba1642d28668bbe470030a85ca88d4a2d157
Python version: 2.7.11
Qt version: 4.8.7
Coin version: 4.0.0a
OCC version: 7.0.0
wmayer
Site Admin
Posts: 16649
Joined: Thu Feb 19, 2009 10:32 am

Re: STEP file read might be crazy long ...

Postby wmayer » Thu Aug 11, 2016 9:02 am

CADExchanger which is also based on OCC claims to load it in a bit more than 10 minutes. When I tried it I could already see parts of the model but it was still loading. After around 15 minutes I had to kill the application because memory usage went up to 4.5 GB.

EDIT: The actual reading of the file is very fast and only takes around 30s. The slow part is the transfer from STEP entities to OCC entities. I wonder if there is a way to do this multi-threaded.
User avatar
sgrogan
Posts: 6166
Joined: Wed Oct 22, 2014 5:02 pm

Re: STEP file read might be crazy long ...

Postby sgrogan » Thu Aug 11, 2016 5:10 pm

wmayer wrote:EDIT: The actual reading of the file is very fast and only takes around 30s. The slow part is the transfer from STEP entities to OCC entities. I wonder if there is a way to do this multi-threaded.
Since CADExchanger 3.2 there is a v2 of the step importer that uses multi-threading. It needs to be turned on in settings. For me there is a large performance increase since v1.
"fight the good fight"
User avatar
vejmarie
Posts: 680
Joined: Mon Jan 04, 2016 4:52 pm
Location: Somewhere between France, USA and Taiwan
Contact:

Re: STEP file read might be crazy long ...

Postby vejmarie » Thu Aug 11, 2016 8:46 pm

sgrogan wrote:
wmayer wrote:EDIT: The actual reading of the file is very fast and only takes around 30s. The slow part is the transfer from STEP entities to OCC entities. I wonder if there is a way to do this multi-threaded.
Since CADExchanger 3.2 there is a v2 of the step importer that uses multi-threading. It needs to be turned on in settings. For me there is a large performance increase since v1.
Hi sgrogan, might be great if we could integrate that feature ;). I didn't even know CADExchanger, but importing STEP is a "standard" behavior in mechanical field and seeing some improvement there might be really good. The good news is that I saved the file in native FreeCAD format and then after reading it is far far much faster (5 minutes instead of 1hour)
User avatar
easyw-fc
Posts: 2915
Joined: Thu Jul 09, 2015 9:34 am

Re: STEP file read might be crazy long ...

Postby easyw-fc » Thu Aug 11, 2016 8:57 pm

vejmarie wrote:
sgrogan wrote:
wmayer wrote:EDIT: The actual reading of the file is very fast and only takes around 30s. The slow part is the transfer from STEP entities to OCC entities. I wonder if there is a way to do this multi-threaded.
Since CADExchanger 3.2 there is a v2 of the step importer that uses multi-threading. It needs to be turned on in settings. For me there is a large performance increase since v1.
Hi sgrogan, might be great if we could integrate that feature ;). I didn't even know CADExchanger, but importing STEP is a "standard" behavior in mechanical field and seeing some improvement there might be really good. The good news is that I saved the file in native FreeCAD format and then after reading it is far far much faster (5 minutes instead of 1hour)
in windows DesignSparks Mechanical is much faster in loading STEP files... it would be great if it would be possible to increase loading speed also for FreeCAD!!! :D
Maurice
User avatar
sgrogan
Posts: 6166
Joined: Wed Oct 22, 2014 5:02 pm

Re: STEP file read might be crazy long ...

Postby sgrogan » Thu Aug 11, 2016 9:00 pm

CADExchanger is a proprietary software but they provide evaluation copies with a 180 day time frame. No production use is permitted by the evaluation license, but as wmayer pointed out the app is based on OCC. Their main (or one of the main developers) is active in the OCCT forums for a long time. I guess the point of my post is it's at least possible to multi-thread step import using OCC and that it works well. From the timing I think OCC >6.9 is required.
http://cadexchanger.com/download/CHANGES.txt
"fight the good fight"