Adding CMake Finder for PySide and Shiboken

Post here if you have re-based and finalised code to integrate into master, which was discussed, agreed to and tested in other forums. You can also submit your PR directly on github.
ezzieyguywuf
Posts: 637
Joined: Tue May 19, 2015 1:11 am

Adding CMake Finder for PySide and Shiboken

Postby ezzieyguywuf » Tue Aug 20, 2019 2:33 am

The pull request can be found here on github.

I have recently merged in the latest master, as well as updated the variable names in the finder scripts to make more sense.

I have been unable to reproduce the errors mentioned in the pull request commentary.
ezzieyguywuf
Posts: 637
Joined: Tue May 19, 2015 1:11 am

Re: Adding CMake Finder for PySide and Shiboken

Postby ezzieyguywuf » Tue Aug 20, 2019 3:50 am

Hm, seems that the travis build is still failing.

I suspect this may have to do with python2 vs. python3.

My development environment uses python3, and with that I have been unable to reproduce this error.

Is it worth it to track down the root cause of this issue using python2?

Or is FreeCAD gradually phasing out support for python2?

If so, can I somehow request the travis build to skip the python2 check?

Edit:

Here is a screenshot showing the portion of the tests that failed on travis working fine on my local build, using Qt5 and python3.
2019-08-19-235307_795x547_scrot.png
2019-08-19-235307_795x547_scrot.png (59.75 KiB) Viewed 807 times
wmayer
Site Admin
Posts: 15228
Joined: Thu Feb 19, 2009 10:32 am

Re: Adding CMake Finder for PySide and Shiboken

Postby wmayer » Tue Aug 20, 2019 2:02 pm

During initialization the error "Cannot load shiboken module" occurred in /usr/local/Mod/Tux/InitGui.py
The problem is that on Linux the shiboken packages lack of the shiboken2 Python module. So, on these systems it's mandatory that we still link against the shared C++ libraries of shiboken2 and PySide2.

Therefore the FreeCAD CMake configuration should support two options:
* allow to link against the C++ libraries as it is now
* allow to dynamically load the Python modules at runtime as suggested by your PR

Also functionality-wise there is one limitation that won't work any more with the Python modules approach:

Code: Select all

ui = FreeCADGui.UiLoader()
w = ui.createWidget("Gui::InputField")
w.show()
w.property("quantity") # when linking against the C++ libs this correctly returns the quantity, otherwise an error is raised
This is because using the C++ API of shiboken you can add your own converter classes but up to my knowledge there is no such mechanism to do it in Python.
ezzieyguywuf
Posts: 637
Joined: Tue May 19, 2015 1:11 am

Re: Adding CMake Finder for PySide and Shiboken

Postby ezzieyguywuf » Tue Aug 20, 2019 2:10 pm

wmayer wrote:
Tue Aug 20, 2019 2:02 pm
on Linux the shiboken packages lack of the shiboken2 Python module.
I'm not sure if this is a Linux-specific issue. Maybe a Debbian-specific issue?

I do know that I was able to compile and install PySide2, which came with Shiboken, on my linux machine in a python virtualenvironment. I am able to fully utilize the FreeCAD Gui, though I am limited in my testing due to some issues with Pivy that I am running into (this is a separate problem though).
wmayer wrote:
Tue Aug 20, 2019 2:02 pm
on these systems it's mandatory that we still link against the shared C++ libraries of shiboken2 and PySide2.
I bet if I can find a way to link against these shared c++ libraries, I should hopefully be able to reproduce the error...Thanks for the hint.
wmayer wrote:
Tue Aug 20, 2019 2:02 pm
Therefore the FreeCAD CMake configuration should support two options:
* allow to link against the C++ libraries as it is now
* allow to dynamically load the Python modules at runtime as suggested by your PR
This seems very reasonable. However, my first step will be to reproduce the error, after which I will spend some time exploring whether or not there are any alternative approaches, especially considering:
wmayer wrote:
Tue Aug 20, 2019 2:02 pm
functionality-wise there is one limitation that won't work any more with the Python modules approach
If the "python modules approach" (as you call it) breaks portions of the code base, then I will either need to abandon the PR altogether or fix the issues whenever I re-submit the PR.

In general, I was unaware that pyside and shiboken were available as shared libraries - I assumed that they were solely available as python modules. Therefore, I have more research and investigation to do before this PR is ready to be merged.

Thankfully, though, the new code accomplishes my short-term goal of allowing me to compile python on my gentoo box :-P
wmayer
Site Admin
Posts: 15228
Joined: Thu Feb 19, 2009 10:32 am

Re: Adding CMake Finder for PySide and Shiboken

Postby wmayer » Tue Aug 20, 2019 2:41 pm

I'm not sure if this is a Linux-specific issue. Maybe a Debbian-specific issue?
It's not Linux-specific but distribution-specific -- it's a bug in the packaging script. I know Debian and all its derivatives are affected but I don't know about other distributions.
If the "python modules approach" (as you call it) breaks portions of the code base, then I will either need to abandon the PR altogether or fix the issues whenever I re-submit the PR.
Personally I prefer the Python module approach as this is much more elegant (just look at the ugly check to find the correct cmake file when trying to link the shared libraries instead).
AFAIK only the Path module somewhere needs to access the Quantity as a property from a QObject. So, it's definitely not the whole application affected by this small limitation.
ezzieyguywuf
Posts: 637
Joined: Tue May 19, 2015 1:11 am

Re: Adding CMake Finder for PySide and Shiboken

Postby ezzieyguywuf » Tue Aug 20, 2019 3:49 pm

wmayer wrote:
Tue Aug 20, 2019 2:41 pm
AFAIK only the Path module somewhere needs to access the Quantity as a property from a QObject. So, it's definitely not the whole application affected by this small limitation.
Well, you’re probably more familiar with the code base than me.

If you’re gut is telling you that there’s few instances that strictly require the shared library, then I’m 100% for making the change across the board

I think my first step will be examining the Debian build packages though and figuring out why they don’t include the python module.

At the very least I need to get Travis build to pass, maybe using my personal ppa...
ezzieyguywuf
Posts: 637
Joined: Tue May 19, 2015 1:11 am

Re: Adding CMake Finder for PySide and Shiboken

Postby ezzieyguywuf » Wed Aug 21, 2019 1:22 pm

Update: I've been able to reproduce the error in an ubuntu virtual machine. I'll see if I can manually build/install pyside/shiboken on the same machine to resolve the issue, and if so try to package up a deb with the "fixed" build.

Curiously, I'm struggling to reproduce the error on my gentoo box, but I believe it is because I'm not quite savvy enough to force FreeCAD into using the shared C library of shiboken that's installed in my home directory. Maybe I can tweak LD_LIBRARY_PATH....
ezzieyguywuf
Posts: 637
Joined: Tue May 19, 2015 1:11 am

Re: Adding CMake Finder for PySide and Shiboken

Postby ezzieyguywuf » Thu Aug 22, 2019 6:41 pm

By way of an update, I've finished setting up a lubuntu 18.04 virtual system to try to recreate the issue. I'm compiling FreeCAD on there now, however in the meantime I did some digging around...

I think the issue may be related to the version of Qt that ships with (l)ubuntu 18.04 LTS. It is using Qt 5.9.5, but I believe pyside2 explicitly requires Qt >= 5.12.

See here for more info.

I'll probably try finding a ppa or something with Qt 5.12 first, to see if that resolves the issue, before I trying mucking about with the pyside2 debs.
wmayer
Site Admin
Posts: 15228
Joined: Thu Feb 19, 2009 10:32 am

Re: Adding CMake Finder for PySide and Shiboken

Postby wmayer » Fri Aug 23, 2019 8:33 am

I think the issue may be related to the version of Qt that ships with (l)ubuntu 18.04 LTS. It is using Qt 5.9.5, but I believe pyside2 explicitly requires Qt >= 5.12
On my virtual Xubuntu 18.04 using PySide2 and Qt 5.9.5 works without problems. The corresponding shiboken2/PySide2 version is 5.11.2

Also on Windows I have built PySide2 with Qt5.6 several years ago and this works without problems using shiboken2/PySide2 version 2.0.0. For my Qt 5.12 version the corresponding shiboken2/PySide2 version is 5.12.2
ezzieyguywuf
Posts: 637
Joined: Tue May 19, 2015 1:11 am

Re: Adding CMake Finder for PySide and Shiboken

Postby ezzieyguywuf » Fri Aug 23, 2019 12:06 pm

wmayer wrote:
Fri Aug 23, 2019 8:33 am
The corresponding shiboken2/PySide2 version is 5.11.2
So shiboken2 is version 5.11.2, built against Qt 5.9.5? Hrm, for some reason I thought the version had to match (though I can't actually recall reading that anywhere...)

Since you're on ubuntu, though, I imagine the shiboken2 python module is not included in the installation?

In other words "python -c 'import shiboken2'" fails, correct? This is the 'issue' that I am trying to resolve, as the FindShiboken2.cmake finder I wrote relies on the python module.