More Weird Behavior from the Draft WB: Memory Leak?

Here's the place for discussion related to coding in FreeCAD, C++ or Python. Design, interfaces and structures.
triplus
Posts: 8605
Joined: Mon Dec 12, 2011 4:45 pm

Re: More Weird Behavior from the Draft WB: Memory Leak?

Postby triplus » Sat Feb 24, 2018 2:42 pm

If i repeat the rectangle test as described earlier. I can confirm the amount of memory increase involved is nowhere near the levels as seen before.
wmayer
Site Admin
Posts: 14585
Joined: Thu Feb 19, 2009 10:32 am

Re: More Weird Behavior from the Draft WB: Memory Leak?

Postby wmayer » Thu Mar 08, 2018 8:11 pm

yorik wrote:
Wed Feb 21, 2018 4:05 pm
I was able to narrow down the memory leak further, it seems to happen inside pivy. For example, this creates a ~30Mb memory increase (takes a couple of minutes to complete):

Code: Select all

from pivy import coin
c = coin.SoCoordinate3()
pts = [[0.0,0.0,0.0],[1.0,1.0,1.0]]
for i in range(1000000):
    c.point.setValues(pts)
Can someone else can test the code snippet above and confirm the memory being eaten?
I can confirm that the memory usage increases by ~30MB but the conclusion is wrong that it's caused by pivy. You achieve the same effect when you simply write:

Code: Select all

r=range(1000000)
when deleting r then the memory usage goes down by ~10MB. But when running the garbage collector then there is no memory overhead at all.

Code: Select all

del r # frees ~10MB
import gc
gc.collect()  # frees the remaining ~20MB
So, if you write

Code: Select all

import gc
from pivy import coin
c = coin.SoCoordinate3()
pts = [[0.0,0.0,0.0],[1.0,1.0,1.0]]
for i in range(1000000):
    c.point.setValues(pts)
    
gc.collect()
you only allocate the 30 MB temporarily. There is also a way to avoid the tmp. consumption completely:

Code: Select all

from pivy import coin
c = coin.SoCoordinate3()
pts = [[0.0,0.0,0.0],[1.0,1.0,1.0]]
i=0
while i<1000000:
    c.point.setValues(pts)
    i=i+1
What remains is that the execution is pretty slow. The same snippet in C++ looks like

Code: Select all

    SbVec3f pts[2] = {SbVec3f(0.0,0.0,0.0),SbVec3f(1.0,1.0,1.0)};
    SoCoordinate3* c = new SoCoordinate3();
    for (int i=0;i<1000000;i++)
        c->point.setValues(0,2,pts);
and it's done more or less immediately.
User avatar
yorik
Site Admin
Posts: 11458
Joined: Tue Feb 17, 2009 9:16 pm
Location: São Paulo, Brazil
Contact:

Re: More Weird Behavior from the Draft WB: Memory Leak?

Postby yorik » Thu Mar 08, 2018 9:16 pm

Wow thanks... that's a real programming lesson!

I thought that when you existed a function, all variables created inside would automatically be deleted by the automatic garbage collection that I always heard python had... And in our case, the problem is not that setValues() runs inside a big loop, but rather that it is called on every mouse move... So in theory the function that contains the setValues() statement is exited each time... I still don't understand...

But it occurs to me now, that actually the whole draft snap queue is triggered on every mouse move, and that you can therefore have several concurrent queues running... That sounds like extremely bad programming :oops: I'll experiment letting it run only one concurrent at a time.
User avatar
yorik
Site Admin
Posts: 11458
Joined: Tue Feb 17, 2009 9:16 pm
Location: São Paulo, Brazil
Contact:

Re: More Weird Behavior from the Draft WB: Memory Leak?

Postby yorik » Thu Mar 08, 2018 9:57 pm

Ok I committed a small change that optimizes snapping a lot, but the memory leak is still there... :( Don't know where it comes from if not from pivy... I'll investigate more then
wmayer
Site Admin
Posts: 14585
Joined: Thu Feb 19, 2009 10:32 am

Re: More Weird Behavior from the Draft WB: Memory Leak?

Postby wmayer » Fri Mar 09, 2018 10:29 am

One remark about the speed issue. I first thought that it could be caused by the SWIG generated code which does quite a lot of type checking and conversions and so on. But when changing the code to:

Code: Select all

from pivy import coin
point = coin.SoMFVec3f()
pts = [[0.0,0.0,0.0],[1.0,1.0,1.0]]
i=0
while i<1000000:
    point.setValues(pts)
    i=i+1
then it only takes a couple of seconds to execute. And when changing the original example to

Code: Select all

from pivy import coin
c = coin.SoCoordinate3()
point = c.point
pts = [[0.0,0.0,0.0],[1.0,1.0,1.0]]
i=0
while i<1000000:
    point.setValues(pts)
    i=i+1
then it also finishes after a couple of seconds. So, the conclusion is that accessing the field of an Inventor node in pivy is very expensive.
User avatar
yorik
Site Admin
Posts: 11458
Joined: Tue Feb 17, 2009 9:16 pm
Location: São Paulo, Brazil
Contact:

Re: More Weird Behavior from the Draft WB: Memory Leak?

Postby yorik » Fri Mar 09, 2018 3:54 pm

Aha that's a very good find... Too much to touch in all Draft to do that now, so close to the release I think, but I'll do that right after. Could give a massive speed gain...