[Solved] huge memory leak when exporting "Flattened SVG"?

[r31415]

Hi,

I think I've found a huge memory leak: about 2GB (!) for each export of a STEP-file to "Flattened SVG".
In addition, the export is quite slow (e.g. ~90s).
Is this already a known issue?

Steps to reproduce:

• Start FreeCAD: memory usage 365 MB ("RES" in top)
• Open stp-file: memory usage 460 MB
• Select part, File -> Export ..., "Flattened SVG (*.svg)": this takes ~90s (!), memory usage 2.4 GB (!)
• Close file: memory usage still 2.4 GB (!)

This can be repeated, until OOM:

• Open stp-file again: memory usage 2.5 GB
• File -> Export ..., "Flattened SVG (*.svg)": this takes ~90s (!), memory usage 4.5 GB
• Close file: memory usage still 4.5 GB (!)
• ...

Version/OS/files:

• FreeCAD 0.19.1
• OS: Linux (NixOS)
• STEP-file of the above example: https://www.motedis.com/daten/Technical ... t%2010.zip

Best regards
Roland

[chrisb]

Confirmed.
After loading step: 380MB
After first export: 2480MB
After 2nd export: 4267MB
After 3rd export: 6252 MB
After closing the file: 6236MB.

[Chris_G]

Confirmed :

- before SVG export : 165 MB
- after 1st export : 2272 MB
- after 2nd export : 4388 MB
- memory not released after file closed

Code: Select all

OS: Manjaro Linux (KDE//usr/share/xsessions/plasma)
Word size of FreeCAD: 64-bit
Version: 0.20.24685 +7 (Git)
Build type: Release
Branch: reflect2
Hash: f37c6b6ff1938853a6d2ea6a5d01f5745aeaa161
Python version: 3.9.2
Qt version: 5.15.2
Coin version: 4.0.1
OCC version: 7.5.0
Locale: French/France (fr_FR)

[wmayer]

git commit 4e4068e44f

[Chris_G]

Thanks.
How did you find this bug ?
I did search in Draft's get_svg() because I thought the leak was happening there.
I didn't think it was down in the C++ code.
What does setNotTracking() do ?
Does it prevent refcount increase of the py object ?

[Kunda1]

@r31415 please prepend [Solved] to this thread by editing the subject line of the first post in this thread. thanks!

[wmayer]

Thanks.
How did you find this bug ?

It was just a test by disabling the define ATTR_TRACKING in PyObjectBase.cpp. If it's off then some parts of the code in __getattro and __setattro are not active that does some tracking.

Additionally, I have a special build where the address sanitizer compiler switch is enabled and this reports a list of function calls that doesn't free allocated memory when exiting FreeCAD. So, I dumped the output to a file when ATTR_TRACKING is active and to another file when it's inactive.
The difference between the two files gives me the affected functions.

I did search in Draft's get_svg() because I thought the leak was happening there.
I didn't think it was down in the C++ code.
What does setNotTracking() do ?

In most of our C++ classes that return a Python object they pass a copy of the wrapped object because passing a reference or pointer is cumbersome and can easily lead to hard to find segmentation faults. But the consequence is that code like this:

Code: Select all

obj.Placement.Base.x = 10.0

has no effect because the change is applied to a tmp. object.

Thus, in the past when changing the placement we had to write code like this:

Code: Select all

placement = obj.Placement
base = placement.Base
base.x = 10.0
placement.Base = base
obj.Placement = placement

that is very unpythonic. Now, in order to support the direct way I implemented the tracking possibility so that when changing the x attribute it notifies the Base attribute that notifies the Placement attribute and this notifies the feature object.

However, there are cases where an attribute is considered read-only (like shape.Vertexes[0].Point) and this then doesn't need to be in the list of tracked objects. In this case e..g the Point attribute is read-only and it's not possible to modify the coordinates of the Vertex but it was still tracked. The result is that there is a cyclic dependency where the reference counter of both objects are incremented and the garbage collector has no chance to find it and thus cannot destroy the Python objects.

Does it prevent refcount increase of the py object ?

Yes.

[Chris_G]

That's very interesting.
Thanks a lot for taking the time for this explanation.

[wmayer]

With the above and some further commits I could avoid any additional memory leak in this use case. However, in other use cases there are still some memory leaks and the problem is that inside the FreeCAD class PyObjectBase a Python attribute keeps a reference to a child attribute and the child attribute keeps a reference to its parent attribute.

So, it ends up in a cyclic dependency which actually isn't a problem if the notification is started. However, if the notification chain isn't started (because the grand-grand-grand-child doesn't get modified) the objects will never be destroyed. To fix this issue it must be avoided to share the attributes in both directions and instead an attribute should only keep a weak reference to its child attribute.

Python supports it to extend own classes by the weak reference interface but unfortunately it only works for plain C structs but not C++ classes (like PyObjectBase). But luckily there is a trick by writing a proxy class that implements the needed interface.

With this branch https://github.com/wwmayer/FreeCAD/comm ... _reference I could finally solve the memory leaks inside PyObjectBase.