memory leak in script?

Need help, or want to share a macro? Post here!
Post Reply
r31415
Posts: 2
Joined: Fri Jan 17, 2020 4:02 pm

memory leak in script?

Post by r31415 » Fri Jan 17, 2020 4:33 pm

Hi,
I've created a script, which loads STP-files and converts them to SVGs. Unfortunately, this script:
  • eats up more and more memory with every STP-file I load/convert, up to several GB
  • on exit, shows an error-message for each loaded STP-file:
    Exception while processing file: FILE1.stp [Cannot load Gui module in console application.]
    Exception while processing file: FILE2.stp [Cannot load Gui module in console application.]
    ...
This is my first FreeCAD-script, so maybe I'm doing something wrong.

So: Can you tell me what I'm doing wrong? Or is this a bug in FreeCAD?

Here's the script:

Code: Select all

#!/usr/bin/env FreeCADCmd
#
# Convert 3D-files to SVG (in the default view) via FreeCAD
#
# :Author: Roland Freikamp
# :Version: 2020-01-17

import os
import sys

import FreeCAD
import Part
import importSVG

print("== Convert 3D-files to SVG")

for filename in sys.argv[2:]:
    svgfile = os.path.splitext(filename)[0].replace(" ","_") + ".svg"

    print("-- %s -> %s" % (filename, svgfile))
    if not os.path.exists(filename):
        print("ERROR: '%s' does not exist." % filename)
        continue
    if os.path.exists(svgfile):
        print("ERROR: SVG-file '%s' already exists." % svgfile)
        continue

    doc = FreeCAD.newDocument("doc")
    p = Part.read(filename)
    f = doc.addObject("Part::Feature", "obj")
    f.Shape = p
    doc.recompute()
    importSVG.export([doc.getObject("obj")], svgfile)

    doc.removeObject("obj")
    FreeCAD.closeDocument("doc")
User avatar
DeepSOIC
Posts: 7291
Joined: Fri Aug 29, 2014 12:45 am
Location: Saint-Petersburg, Russia

Re: memory leak in script?

Post by DeepSOIC » Fri Jan 17, 2020 5:20 pm

r31415 wrote:
Fri Jan 17, 2020 4:33 pm
Can you tell me what I'm doing wrong?
Seems good to me, nothing obvious to cause a memory leak.

What is your FreeCAD version, btw? (see IMPORTANT: Please read first before asking for help for how to post it)
openBrain
Posts: 3087
Joined: Fri Nov 09, 2018 5:38 pm

Re: memory leak in script?

Post by openBrain » Fri Jan 17, 2020 6:08 pm

I think your whole export processing could be reduced to something like:

Code: Select all

importSVG.export(Import.open(filename), svgfile)
You have to import the 'Import' module, but no need to create a document and/or populate it with object. ;)
wmayer
Site Admin
Posts: 15274
Joined: Thu Feb 19, 2009 10:32 am

Re: memory leak in script?

Post by wmayer » Sat Jan 18, 2020 10:56 am

on exit, shows an error-message for each loaded STP-file:
This is because when passing several file names to the executable then it tries to handle them one by one. This means the application first runs your script which internally already handles the further arguments and after it's finished it tries to handle the STEP file.

Now there is a mistake in how the STEP file format is added because it's registered for the ImportGui module but for a GUI-less application this makes not much sense. So, this is something that should still be improved in FreeCAD.

Nevertheless in order to get rid of this issue just add quit() at the end of your script because then the application terminates after having executed it.
eats up more and more memory with every STP-file I load/convert, up to several GB
Your script looks fine but under the hood a lot of things happen. If there is a memory leak somewhere then most likely when loading the STEP or when converting it. You can adjust the script to check if one of the two is the culprit:
  • Only load STEP files without converting them. If the RAM usage increases constantly then this is the problem.
  • Only load one STEP a single time and convert it several times. If the RAM usage increases constantly then this is the problem.
wmayer
Site Admin
Posts: 15274
Joined: Thu Feb 19, 2009 10:32 am

Re: memory leak in script?

Post by wmayer » Sat Jan 18, 2020 2:33 pm

wmayer wrote:
Sat Jan 18, 2020 10:56 am
Now there is a mistake in how the STEP file format is added because it's registered for the ImportGui module but for a GUI-less application this makes not much sense. So, this is something that should still be improved in FreeCAD.
This git commit 120b69c74f7 fixes the file type issue. So, when using the command line version it won't try to load the GUI module any more.
r31415
Posts: 2
Joined: Fri Jan 17, 2020 4:02 pm

Re: memory leak in script?

Post by r31415 » Tue Jan 21, 2020 3:28 pm

Thanks for your answers.
  • memory leak:

    Code: Select all

    importSVG.export(...)
    eats up the memory. Without this command, memory usage is ok.

    Loading only one stp-file, and running importSVG.export(...) in a loop, also eats up memory.
    It looks like importSVG.export never frees the memory again.

    Simplified demo-script (but beware: use a memory monitor and be ready to hit Ctrl-C to abort the script before it eats up too much memory!):

    Code: Select all

    import sys
    
    import FreeCAD
    import Part
    import importSVG
    import Import
    
    filename = sys.argv[2]
    
    doc = FreeCAD.newDocument("doc")
    p = Part.read(filename)
    f = doc.addObject("Part::Feature", "obj")
    f.Shape = p
    doc.recompute()
    obj = doc.getObject("obj")
    
    for i in range(0,10):
        print("--", i)
        importSVG.export([obj], "%d.svg" % i)
    
    doc.removeObject("obj")
    FreeCAD.closeDocument("doc")
    quit()
  • error-messages on exit:
    wmayer wrote:
    Sat Jan 18, 2020 10:56 am
    Nevertheless in order to get rid of this issue just add quit() at the end of your script because then the application terminates after having executed it.
    Adding quit() or sys.exit(0) removes the error-messages at the end.
  • simplified script:
    openBrain wrote:
    Fri Jan 17, 2020 6:08 pm
    I think your whole export processing could be reduced to something like:

    Code: Select all

    importSVG.export(Import.open(filename), svgfile)
    You have to import the 'Import' module, but no need to create a document and/or populate it with object. ;)
    Unfortunately not; I've tried this, but this results in the error:
    Exception while processing file: /home/user/convert_stp_to_svg.py ['tuple' object has no attribute 'isDerivedFrom']
    Probably this works for converting between 3D-formats, but not for exporting to 2D.
  • FreeCAD-version: 0.18.3
    Details:
    OS: NixOS 19.09
    Word size of OS: 64-bit
    Word size of FreeCAD: 64-bit
    Version: 0.18.Unknown
    Build type: Release
    Python version: 3.7.5
    Qt version: 5.12.3
    Coin version: 4.0.0a
    OCC version: 7.3.0
    Locale: English/Denmark (en_DK)

    I could try this with 0.18.4 or 0.19_pre or the latest commit, if this would help.
Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest