Porting to python3

Here's the place for discussion related to coding in FreeCAD, C++ or Python. Design, interfaces and structures.
User avatar
Kunda1
Posts: 635
Joined: Thu Jan 05, 2017 9:03 pm

Re: Porting to python3

Postby Kunda1 » Sat Jan 28, 2017 11:42 am

(This is not meant to hijack the thread.)
Quick question: there is an open ticket on the tracker issue #995 named "Python3 compatibility" assigned to @yorik which is not tracking this thread of development. Please advise.
wmayer
Site Admin
Posts: 11234
Joined: Thu Feb 19, 2009 10:32 am

Re: Porting to python3

Postby wmayer » Sat Jan 28, 2017 5:52 pm

looo wrote:finally a working code. With this the part-design-gui tests are passing.

https://github.com/looooo/FreeCAD/commi ... 8bdcf7d4bc

I compiled the py3-21 and reverted this change to see what has happened.
When running this snippet

Code: Select all

doc=App.newDocument()
body=doc.addObject("PartDesign::Body","Body")
body.addObject

it goes through PartDesign::BodyPy::_getattro where PyObject_GenericGetAttr then of course finds "addObject" since we added it to the type object. It then immediately returns to the calling instance __getattro which inspects the PyCFunction and raises an error because no object is bound to it.
wmayer
Site Admin
Posts: 11234
Joined: Thu Feb 19, 2009 10:32 am

Re: Porting to python3

Postby wmayer » Sat Jan 28, 2017 6:23 pm

In your fix you do this:

Code: Select all

    for (unsigned int i=0; i< sizeof(this->Methods); i++)
    {
        PyMethodDef* meth = & Methods[i];
        if (Methods[i].ml_name == attr)
            return PyCFunction_New(meth, this);
    }

which won't work because you must not compare C strings with "==".

Instead you should do:

Code: Select all

    PyMethodDef *ml = Methods;
    for (; ml->ml_name != NULL; ml++) {
        if (name[0] == ml->ml_name[0] &&
            strcmp(name+1, ml->ml_name+1) == 0)
            return PyCFunction_New(ml, self);
    }

which is also faster.
looo
Posts: 706
Joined: Mon Nov 11, 2013 5:29 pm

Re: Porting to python3

Postby looo » Sat Jan 28, 2017 6:45 pm

wmayer wrote:it goes through PartDesign::BodyPy::_getattro where PyObject_GenericGetAttr then of course finds "addObject" since we added it to the type object. It then immediately returns to the calling instance __getattro which inspects the PyCFunction and raises an error because no object is bound to it.


Thanks for looking at this.
So this means we can also use the PyObject_GenericGetAttr and bind the object to the function?

The fix I have pushed to the branch introduce some problems with fem-tests. This one is really difficult, because it only happens when I run TestApp.All. Running only the fem-tests is not leading to a crash and fem tests are succeeding.

I don't know if this trace-back is useful, but it shows the crash is definetly coupled to the _getattro function:

Code: Select all

#0  0x00007fff6cbf3f50 in Fem::FemMeshPy::_getattro (this=0x83db4d0, attro=0x7fffb25f2960) from ~/anaconda/envs/fc_py3/lib/Fem.so
#1  0x00007ffff5fa9fea in Base::PyObjectBase::__getattro (obj=0x83db4d8, attro=0x7fffb25f2960) from ~/anaconda/envs/fc_py3/bin/../lib/libFreeCADBase.so
#2  0x00007ffff5a6e9bb in PyEval_EvalFrameEx (f=f@entry=0x8434548, throwflag=throwflag@entry=0) at Python/ceval.c:2758
#3  0x00007ffff5a71a70 in fast_function (nk=<optimized out>, na=<optimized out>, n=1, pp_stack=0x7fffffff9fb0, func=<optimized out>) at Python/ceval.c:4818
#4  call_function (oparg=<optimized out>, pp_stack=0x7fffffff9fb0) at Python/ceval.c:4745
#5  PyEval_EvalFrameEx (f=f@entry=0x7fff421e1408, throwflag=throwflag@entry=0) at Python/ceval.c:3251
#6  0x00007ffff5a71a70 in fast_function (nk=<optimized out>, na=<optimized out>, n=1, pp_stack=0x7fffffffa130, func=<optimized out>) at Python/ceval.c:4818
#7  call_function (oparg=<optimized out>, pp_stack=0x7fffffffa130) at Python/ceval.c:4745
#8  PyEval_EvalFrameEx (f=f@entry=0x6d78048, throwflag=throwflag@entry=0) at Python/ceval.c:3251
#9  0x00007ffff5a74036 in _PyEval_EvalCodeWithName (_co=0x7fffbcbe2d20, globals=<optimized out>, locals=locals@entry=0x0, args=args@entry=0x7fff6c010060, argcount=2, kws=kws@entry=0x7ffff7fb4060,
    kwcount=kwcount@entry=0, defs=defs@entry=0x7fffbcbc1220, defcount=defcount@entry=1, kwdefs=kwdefs@entry=0x0, closure=closure@entry=0x0, name=name@entry=0x0, qualname=qualname@entry=0x0)
    at Python/ceval.c:4033


ps.: I think the answer is above. :D
wmayer
Site Admin
Posts: 11234
Joined: Thu Feb 19, 2009 10:32 am

Re: Porting to python3

Postby wmayer » Sat Jan 28, 2017 8:34 pm

So this means we can also use the PyObject_GenericGetAttr and bind the object to the function?

No, because the binding of the object can only be done in ExtensionContainerPy::getCustomAttributes.
looo
Posts: 706
Joined: Mon Nov 11, 2013 5:29 pm

Re: Porting to python3

Postby looo » Sat Jan 28, 2017 9:32 pm

thanks for the explanation.
maybe some day I will understand it. c++ is quite complex.

Now nearly all tests are passing. One more difficult is remaining:
dynamic module does not define module export function (PyInit_ReverseEngineering)


and also the Path workbench has many small python3-incompatibilities. and many more easy ones:

Code: Select all

 
File "Mod/Path/PathScripts/PathDressupDogbone.py", line 954
    print i.Group
          ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Path/PathScripts/example_post.py", line 45
    print "This script is unable to write more than one Path object"
                                                                   ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Arch/import3DS.py", line 109
    print "Skipping object without vertices array: ",d_nobj.obj
                                                   ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Path/PathScripts/PathCompoundExtended.py", line 60
    print 'check order'
                      ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Path/PathScripts/PathPostProcessor.py", line 38
    exec "import %s as current_post" % postname
                                   ^
SyntaxError: Missing parentheses in call to 'exec'

  File "Mod/Path/PathScripts/PathKurveUtils.py", line 145
    print "in profile: 151"
                          ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Path/PathScripts/centroid_post.py", line 88
    print "No machine found in this selection"
                                             ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Path/PathScripts/opensbp_post.py", line 84
    print s
          ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Path/PathScripts/dumper_post.py", line 55
    print "the object " + obj.Name + " is not a path. Please select only path and Compounds."
                      ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Path/PathScripts/opensbp_pre.py", line 84
    print "preprocessing..."
                           ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Fem/FemMesh2Mesh.py", line 134
    print 'found a last Face: ', faceCodeList[actFaceIdx]
                              ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Path/PathScripts/PathPost.py", line 193
    print "in activated %s" %(selected)
                          ^
SyntaxError: invalid syntax

  File "Mod/Path/PathScripts/linuxcnc_post.py", line 121
    print "the object " + obj.Name + " is not a path. Please select only path and Compounds."
                      ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Path/PathScripts/generic_post.py", line 288
    print "the object " + obj.Name + " is not a path. Please select only path and Compounds."
                      ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Arch/Dice3DS/dom3ds.py", line 297
    except (File3dsFormatError, FBufError), fe:
                                          ^
SyntaxError: invalid syntax

  File "Mod/Path/PathScripts/PathPocket.py", line 157
    print "this base is: " + str(baselist)
                         ^
SyntaxError: invalid syntax

  File "Mod/Arch/importIFC.py", line 625
    print "failed to compute placement ",
                                       ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Arch/importSH3D.py", line 123
    if DEBUG: print "Creating wall: ",name
                                    ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Path/PathScripts/PathToolLibraryManager.py", line 257
    except Exception, e:
                    ^
SyntaxError: invalid syntax

  File "Mod/Path/PathScripts/PathUtils.py", line 623
    print form.cboProject.currentText()
             ^
SyntaxError: invalid syntax

  File "Mod/Path/PathScripts/PathSurface.py", line 148
    print "    loop ", nloop, " with ", len(loop), " points"
                    ^
SyntaxError: invalid syntax

  File "Mod/Path/PathScripts/PathSanity.py", line 53
    print "Checking: " + item.Label
                     ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Path/PathScripts/PathDressupDragknife.py", line 454
    print i.Group
          ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Fem/FemToolsZ88.py", line 104
    print z88_std_location
                         ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Path/PathScripts/PathDrilling.py", line 307
    print baselist
                 ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Path/PathScripts/PathDressup.py", line 93
    print i.Group
          ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Fem/importZ88Mesh.py", line 389
    print elem_length
                    ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Spreadsheet/importXLSX.py", line 370
    print 'process Strings: '
                            ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Path/PathScripts/PathMillFace.py", line 214
    print "in execute"
                     ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Path/PathScripts/dynapath_post.py", line 117
    print "the object " + obj.Name + " is not a path. Please select only path and Compounds."
                      ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Path/PathScripts/PathLog.py", line 85
    def _log(level, (module, line, func), msg):
                    ^
SyntaxError: invalid syntax

  File "Mod/Material/MaterialEditor.py", line 251
    print "DrawingPatterns not found"
                                    ^
SyntaxError: Missing parentheses in call to 'print'



anybody volunteering? :D
Jee-Bee
Posts: 615
Joined: Tue Jun 16, 2015 10:32 am
Location: Netherlands

Re: Porting to python3

Postby Jee-Bee » Sun Jan 29, 2017 6:35 pm

looo wrote:thanks for the explanation.
maybe some day I will understand it. c++ is quite complex.

I saw a link basic c++ i wan't to do it eighter https://developers.google.com/edu/c++/
wmayer
Site Admin
Posts: 11234
Joined: Thu Feb 19, 2009 10:32 am

Re: Porting to python3

Postby wmayer » Sun Jan 29, 2017 9:09 pm

dynamic module does not define module export function (PyInit_ReverseEngineering)

In https://github.com/looooo/FreeCAD/blob/ ... g.cpp#L748 replace

Code: Select all

PyMODINIT_FUNC initReverseEngineering()

with

Code: Select all

PyMOD_INIT_FUNC(ReverseEngineering)
User avatar
kkremitzki
Posts: 326
Joined: Thu Mar 03, 2016 9:52 pm
Location: Texas

Re: Porting to python3

Postby kkremitzki » Mon Jan 30, 2017 3:19 am

I adapted Bernd's script, tested on Ubuntu 16.04, to build FreeCAD 0.17 + OCCT 7.0.0 + VTK 7.0.0 + Netgen 5.3.1 + libMED, from here, and added the compile instructions from this thread. The script, available here:

https://gist.github.com/kkremitzki/324f ... cad_all-sh

compiles and installs from the py3-21 branch in looooo's FreeCAD repo against Python 3.5.

Hopefully this can make it easier for other people to test the Python 3 port. Let me know if anyone has any feedback!
User avatar
kkremitzki
Posts: 326
Joined: Thu Mar 03, 2016 9:52 pm
Location: Texas

Re: Porting to python3

Postby kkremitzki » Mon Jan 30, 2017 5:18 am

looo wrote:and also the Path workbench has many small python3-incompatibilities. and many more easy ones:

Code: Select all

 
File "Mod/Path/PathScripts/PathDressupDogbone.py", line 954
    print i.Group
          ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Path/PathScripts/example_post.py", line 45
    print "This script is unable to write more than one Path object"
                                                                   ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Arch/import3DS.py", line 109
    print "Skipping object without vertices array: ",d_nobj.obj
                                                   ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Path/PathScripts/PathCompoundExtended.py", line 60
    print 'check order'
                      ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Path/PathScripts/PathPostProcessor.py", line 38
    exec "import %s as current_post" % postname
                                   ^
SyntaxError: Missing parentheses in call to 'exec'

  File "Mod/Path/PathScripts/PathKurveUtils.py", line 145
    print "in profile: 151"
                          ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Path/PathScripts/centroid_post.py", line 88
    print "No machine found in this selection"
                                             ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Path/PathScripts/opensbp_post.py", line 84
    print s
          ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Path/PathScripts/dumper_post.py", line 55
    print "the object " + obj.Name + " is not a path. Please select only path and Compounds."
                      ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Path/PathScripts/opensbp_pre.py", line 84
    print "preprocessing..."
                           ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Fem/FemMesh2Mesh.py", line 134
    print 'found a last Face: ', faceCodeList[actFaceIdx]
                              ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Path/PathScripts/PathPost.py", line 193
    print "in activated %s" %(selected)
                          ^
SyntaxError: invalid syntax

  File "Mod/Path/PathScripts/linuxcnc_post.py", line 121
    print "the object " + obj.Name + " is not a path. Please select only path and Compounds."
                      ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Path/PathScripts/generic_post.py", line 288
    print "the object " + obj.Name + " is not a path. Please select only path and Compounds."
                      ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Arch/Dice3DS/dom3ds.py", line 297
    except (File3dsFormatError, FBufError), fe:
                                          ^
SyntaxError: invalid syntax

  File "Mod/Path/PathScripts/PathPocket.py", line 157
    print "this base is: " + str(baselist)
                         ^
SyntaxError: invalid syntax

  File "Mod/Arch/importIFC.py", line 625
    print "failed to compute placement ",
                                       ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Arch/importSH3D.py", line 123
    if DEBUG: print "Creating wall: ",name
                                    ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Path/PathScripts/PathToolLibraryManager.py", line 257
    except Exception, e:
                    ^
SyntaxError: invalid syntax

  File "Mod/Path/PathScripts/PathUtils.py", line 623
    print form.cboProject.currentText()
             ^
SyntaxError: invalid syntax

  File "Mod/Path/PathScripts/PathSurface.py", line 148
    print "    loop ", nloop, " with ", len(loop), " points"
                    ^
SyntaxError: invalid syntax

  File "Mod/Path/PathScripts/PathSanity.py", line 53
    print "Checking: " + item.Label
                     ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Path/PathScripts/PathDressupDragknife.py", line 454
    print i.Group
          ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Fem/FemToolsZ88.py", line 104
    print z88_std_location
                         ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Path/PathScripts/PathDrilling.py", line 307
    print baselist
                 ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Path/PathScripts/PathDressup.py", line 93
    print i.Group
          ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Fem/importZ88Mesh.py", line 389
    print elem_length
                    ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Spreadsheet/importXLSX.py", line 370
    print 'process Strings: '
                            ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Path/PathScripts/PathMillFace.py", line 214
    print "in execute"
                     ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Path/PathScripts/dynapath_post.py", line 117
    print "the object " + obj.Name + " is not a path. Please select only path and Compounds."
                      ^
SyntaxError: Missing parentheses in call to 'print'

  File "Mod/Path/PathScripts/PathLog.py", line 85
    def _log(level, (module, line, func), msg):
                    ^
SyntaxError: invalid syntax

  File "Mod/Material/MaterialEditor.py", line 251
    print "DrawingPatterns not found"
                                    ^
SyntaxError: Missing parentheses in call to 'print'



anybody volunteering? :D

PR submitted :)