Porting to python3

Here's the place for discussion related to coding in FreeCAD, C++ or Python. Design, interfaces and structures.
User avatar
looo
Posts: 3301
Joined: Mon Nov 11, 2013 5:29 pm

Re: Porting to python3

Postby looo » Thu Dec 22, 2016 7:52 pm

btw.': shouldn't travis build with python2 and python3? https://github.com/looooo/FreeCAD/blob/ ... is.yml#L32
User avatar
saso
Posts: 1490
Joined: Fri May 16, 2014 1:14 pm
Contact:

Re: Porting to python3

Postby saso » Thu Dec 22, 2016 9:32 pm

looo wrote:btw.': shouldn't travis build with python2 and python3? https://github.com/looooo/FreeCAD/blob/ ... is.yml#L32
It will have to be something like this
https://github.com/sasobadovinac/FreeCA ... is.yml#L44
https://travis-ci.org/sasobadovinac/Fre ... /186190920

About the build problem, I did not have time today to look more in to it. All I did was, I made a new branch test_py3 from FreeCAD:master, it started to build ok, then I pulled in your branch py3-21 and it fails with those errors...
User avatar
looo
Posts: 3301
Joined: Mon Nov 11, 2013 5:29 pm

Re: Porting to python3

Postby looo » Thu Dec 22, 2016 10:08 pm

ah thanks for the information. Python3 needs for sure some more love. The problem for me at the moment is, that conda doesn't allow to reuse a build. So I have to build the full tree all the time to test with python3.

With the std. ubuntu python3 librariues I get this error:

Code: Select all

make[2]: *** Keine Regel vorhanden, um das Ziel „/usr/lib/x86_64-linux-gnu/libproj.so“, 
benötigt von „lib/libSMDS.so“, zu erstellen.  Schluss.
User avatar
looo
Posts: 3301
Joined: Mon Nov 11, 2013 5:29 pm

Re: Porting to python3

Postby looo » Sat Dec 24, 2016 11:32 am

@ickby
the extension added a new python3 error. Maybe you can have a look at this file: https://github.com/FreeCAD/FreeCAD/blob ... p.cpp#L126 The Py_FindMethod isn't available for python3. I couldn't figure out how to solve this. Maybe this can be of any help:
https://groups.google.com/forum/#!topic ... UoKr2fI3p8
wmayer
Site Admin
Posts: 15756
Joined: Thu Feb 19, 2009 10:32 am

Re: Porting to python3

Postby wmayer » Sat Dec 24, 2016 3:13 pm

This is the implementation of Py_FindMethod:

Code: Select all

/* Find a method in a method chain */

PyObject *
Py_FindMethodInChain(PyMethodChain *chain, PyObject *self, const char *name)
{
    if (name[0] == '_' && name[1] == '_') {
        if (strcmp(name, "__methods__") == 0) {
            if (PyErr_WarnPy3k("__methods__ not supported in 3.x",
                               1) < 0)
                return NULL;
            return listmethodchain(chain);
        }
        if (strcmp(name, "__doc__") == 0) {
            const char *doc = self->ob_type->tp_doc;
            if (doc != NULL)
                return PyString_FromString(doc);
        }
    }
    while (chain != NULL) {
        PyMethodDef *ml = chain->methods;
        for (; ml->ml_name != NULL; ml++) {
            if (name[0] == ml->ml_name[0] &&
                strcmp(name+1, ml->ml_name+1) == 0)
                /* XXX */
                return PyCFunction_New(ml, self);
        }
        chain = chain->link;
    }
    PyErr_SetString(PyExc_AttributeError, name);
    return NULL;
}

/* Find a method in a single method list */

PyObject *
Py_FindMethod(PyMethodDef *methods, PyObject *self, const char *name)
{
    PyMethodChain chain;
    chain.methods = methods;
    chain.link = NULL;
    return Py_FindMethodInChain(&chain, self, name);
}
In this case all what it actually needs to do is the part of the while loop:

Code: Select all

    while (chain != NULL) {
        PyMethodDef *ml = chain->methods;
        for (; ml->ml_name != NULL; ml++) {
            if (name[0] == ml->ml_name[0] &&
                strcmp(name+1, ml->ml_name+1) == 0)
                /* XXX */
                return PyCFunction_New(ml, self);
        }
        chain = chain->link;
    }
User avatar
looo
Posts: 3301
Joined: Mon Nov 11, 2013 5:29 pm

Re: Porting to python3

Postby looo » Sat Dec 24, 2016 3:56 pm

thanks for the hint.
I have found another library which had this kind of problem. Now I have tried to use there way, but I am not sure if this is the right solution.

Code: Select all

#if PY_MAJOR_VERSION >= 3
        PyObject *nameobj = PyUnicode_FromString(attr);
        func = PyObject_GenericGetAttr((PyObject*) obj, nameobj);
#else
        func = Py_FindMethod(meth, obj, attr);
#endif
copied from here:
https://github.com/jnthntatum/pcapy/blo ... /bpfobj.cc

After resolving a few other errors I am now stuck with a annoying boost linking error.

Code: Select all

[  9%] Linking CXX shared library ../../lib/libFreeCADApp.so
CMakeFiles/FreeCADApp.dir/PropertyPythonObject.cpp.o: In Funktion `bool boost::regex_search<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >, char, boost::regex_traits<char, boost::cpp_regex_traits<char> > >(__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, __gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::match_results<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > >&, boost::basic_regex<char, boost::regex_traits<char, boost::cpp_regex_traits<char> > > const&, boost::regex_constants::_match_flags, __gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >) [clone .constprop.152]':
PropertyPythonObject.cpp:(.text+0x35d5): Nicht definierter Verweis auf `boost::re_detail_106200::perl_matcher<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >, boost::regex_traits<char, boost::cpp_regex_traits<char> > >::construct_init(boost::basic_regex<char, boost::regex_traits<char, boost::cpp_regex_traits<char> > > const&, boost::regex_constants::_match_flags)'
CMakeFiles/FreeCADApp.dir/PropertyPythonObject.cpp.o: In Funktion `boost::re_detail_106200::perl_matcher<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >, boost::regex_traits<char, boost::cpp_regex_traits<char> > >::match_match()':
........................
wmayer
Site Admin
Posts: 15756
Joined: Thu Feb 19, 2009 10:32 am

Re: Porting to python3

Postby wmayer » Sat Dec 24, 2016 4:42 pm

I have found another library which had this kind of problem. Now I have tried to use there way, but I am not sure if this is the right solution.
The Py_FindMethod either finds a function of the object or nothing while PyObject_GenericGetAttr finds any attribute of this name. Not sure if this causes any problems with the extension framework.

Btw, PyUnicode_FromString returns a new object and after using PyObject_GenericGetAttr you must Py_DECREF(nameobj) otherwise you create a memory leak.
User avatar
looo
Posts: 3301
Joined: Mon Nov 11, 2013 5:29 pm

Re: Porting to python3

Postby looo » Sun Dec 25, 2016 8:55 pm

thanks, I have added the decref.

my previous reported error was to due some gcc updates... with gcc4.8 this is gone.
But now I am stuck with some python3 problem in this line:

https://github.com/looooo/FreeCAD/blob/ ... t.cpp#L189

Code: Select all

/home/lo/projects/FreeCAD/src/Mod/Part/App/AppPart.cpp:189:5: error: ‘Py_TPFLAGS_HAVE_CLASS’ was not declared in this scope
     Py_TPFLAGS_HAVE_CLASS,        /*tp_flags */
Maybe someone can find the thread about the problems with generator files and spreadsheet wb. It was at the beginning of october this year.
User avatar
looo
Posts: 3301
Joined: Mon Nov 11, 2013 5:29 pm

Re: Porting to python3

Postby looo » Tue Dec 27, 2016 10:39 am

There is something wrong with the generators in this branch.
Travis reports about generators cannot create files:

Code: Select all

Traceback (most recent call last):
  File "/home/travis/build/sasobadovinac/FreeCAD/src/Tools/generate.py", line 29, in <module>
    import generateTemplates.templateModule
  File "/home/travis/build/sasobadovinac/FreeCAD/src/Tools/generateTemplates/templateModule.py", line 5, in <module>
    from . import template, templateModuleApp,templateModuleGui
  File "/home/travis/build/sasobadovinac/FreeCAD/src/Tools/generateTemplates/templateModuleApp.py", line 5, in <module>
    from . import template, templateModuleAppMain, templateModuleAppFeature
  File "/home/travis/build/sasobadovinac/FreeCAD/src/Tools/generateTemplates/templateModuleAppMain.py", line 7, in <module>
    import generateBase.generateTools
  File "/home/travis/build/sasobadovinac/FreeCAD/src/Tools/generateBase/generateTools.py", line 80
    exec(stat, self.globals, self.locals)
SyntaxError: unqualified exec is not allowed in function 'copyblock' it contains a nested function with free variables
locally I cannot reproduce this error. Somehow this must be related to the change of the exec (exec ... -> exec(...)). No idea why I cannot reproduce this locally. Maybe because of different python versions. Also the osx builds doesn't have this error on travis. But this is maybe caused by the cmake caching.

With python3 on a local machine there is this error:

Code: Select all

Nicht definierter Verweis auf `Spreadsheet::PropertyRowHeightsPy::PropertyRowHeightsPy(Spreadsheet::PropertyRowHeights*, _typeobject*)'
The generators do not create the needed files. But there isn't any output saying what is going on. How can this be possible?
User avatar
yorik
Site Admin
Posts: 11864
Joined: Tue Feb 17, 2009 9:16 pm
Location: São Paulo, Brazil
Contact:

Re: Porting to python3

Postby yorik » Tue Dec 27, 2016 12:50 pm

Rebasing right now is proving really difficult, in the middle of the Qt5 migration. You'd better wait a bit until it finishes, you'll spare yourself a couple of white hairs I think. I should have more time to work on this too now.