toolmanager bombed out , all tools gone.

Here's the place for discussion related to CAM/CNC and the development of the Path module.
User avatar
freman
Posts: 775
Joined: Tue Nov 27, 2018 10:30 pm

Re: toolmanager bombed out , all tools gone.

Postby freman » Tue Oct 08, 2019 12:45 pm

Wow, thanks for dealing with so many questions at the same time.
Look at your call stack. You have a mix-up of Py2 and Py3 which is not going to work. Apparently the problem is the existence of PySide which is built for Py2. You have to un-install it completely and instead install PySide2.
yep, that is why I posted the bit of the call stack and cmake saying it was using python 2.7. ( I've seen cmake produce erroneous output so not sure it that is really happening ). There is no PySide2 in Fedora, there are py2 and py3 versions of it , I have both installed. It is possible that cmake is not picking the right one.

Code: Select all

dnf search pyside
Last metadata expiration check: 0:02:49 ago on Tue 08 Oct 2019 13:25:29 BST.
================================================= Name & Summary Matched: pyside ==================================================
pyside-tools.x86_64 : Development tools for PySide
pyside-tools.x86_64 : Development tools for PySide
python2-pyudev-pyside.noarch : PySide integration for pyudev
python-pyside-devel.x86_64 : Development files for python-pyside
python-pyside-devel.i686 : Development files for python-pyside
python-pyside-devel.x86_64 : Development files for python-pyside
====================================================== Name Matched: pyside =======================================================
python2-pyside.x86_64 : Python bindings for Qt4
python3-pyside.x86_64 : Python bindings for Qt4
python2-pyside.i686 : Python bindings for Qt4
python2-pyside.x86_64 : Python bindings for Qt4
python3-pyside.i686 : Python bindings for Qt4
python3-pyside.x86_64 : Python bindings for Qt4
===================================================== Summary Matched: pyside =====================================================
python2-QtAwesome.noarch : FontAwesome icons in PyQt and PySide applications
python3-QtAwesome.noarch : FontAwesome icons in PyQt and PySide applications


bash-4.4#dnf install python3-pyside  python2-pyside  python-pyside-devel
Last metadata expiration check: 0:04:15 ago on Tue 08 Oct 2019 13:25:29 BST.
Package python3-pyside-1.2.4-7.fc29.x86_64 is already installed.
Package python2-pyside-1.2.4-7.fc29.x86_64 is already installed.
Package python-pyside-devel-1.2.4-7.fc29.x86_64 is already installed.
Dependencies resolved.
Nothing to do.
Complete!
Nevertheless, if someone runs into a Py2 or Qt4 problem that can be easily fixed I am still open to make a fix for it.
dub has done some nice work on adding coolant stuff. How hard would it be to make this read and write the tooltables without causing serious breakage? It seems to be mostly basic UI changes, this probably should not present much of a problem to work on py2.
The problem is that the executable "python" on most systems still points to python2. You have to manually change this entry to python3.
I'm not sure what you are suggesting here, but if it is that I start messing with the carefully crafted parallel installation of py2 and py3 libs that Fedora devs have put together, that's a no-no. I don't mind having an experimental dev version of FC, but I'm not allowed to undermine the stability of the system as a whole just to make it build.

If the manual change means the cmake option, I did that but it does not seem to be picking up this one instance where it is still pulling in py 2.7 code.

I need to understand exactly why the python 2.7 stuff is getting linked when the relevant py3 versions are in place. I suspect cmake has not been set up properly or is assuming a Debian* specific configuration which does not work on other distros.

Code: Select all

/lib64/python3.7/site-packages/PySide/QtCore.so
I seems that cmake is not using this when the py3 cmake option is given.

dubstar:
you could always ask in the install / compile part of the forum: https://forum.freecadweb.org/viewforum.php?f=4
I did ;)
https://forum.freecadweb.org/viewtopic.php?f=4&t=39890
Last edited by freman on Tue Oct 08, 2019 1:15 pm, edited 1 time in total.
wmayer
Site Admin
Posts: 14999
Joined: Thu Feb 19, 2009 10:32 am

Re: toolmanager bombed out , all tools gone.

Postby wmayer » Tue Oct 08, 2019 1:07 pm

dubstar-04 wrote:
Tue Oct 08, 2019 12:26 pm
wmayer wrote:
Tue Oct 08, 2019 12:04 pm
Be aware that there is a bug in the 18.x tool table that prevented tools being saved. That was fixed here:
https://github.com/FreeCAD/FreeCAD/comm ... 33b97ed8aa
Is it hard to back port this to 0.18?
I added the commit to the 0.18 backport thread: https://forum.freecadweb.org/viewtopic. ... 80#p330071

Is there something I can do to help get it included?

Thanks,

Dan
Thanks. I will add it to the 0.18 branch very soon.
wmayer
Site Admin
Posts: 14999
Joined: Thu Feb 19, 2009 10:32 am

Re: toolmanager bombed out , all tools gone.

Postby wmayer » Tue Oct 08, 2019 1:38 pm

There is no PySide2 in Fedora, there are py2 and py3 versions of it , I have both installed.
Then try the combination Py3/Qt4.
It is possible that cmake is not picking the right one.
This is very well possible. With the cmake files offered by shiboken or PySide the way how to include them into the build process is very ugly and not so straightforward as for most other libraries. So, IMO it's up to the upstream developers of shiboken/PySide to provide proper CMake files that can be easily used.
dub has done some nice work on adding coolant stuff. How hard would it be to make this read and write the tooltables without causing serious breakage? It seems to be mostly basic UI changes, this probably should not present much of a problem to work on py2.
From the call stack we know where PyCXX raises the error but we need to locate the line of code where the Bytes object is created (it must be C++ code) and make it compatible with Py2. However, I can't say how difficult this will be.
I'm not sure what you are suggesting here, but if it is that I start messing with the carefully crafted parallel installation of py2 and py3 libs that Fedora devs have put together, that's a no-no. I don't mind having an experimental dev version of FC, but I'm not allowed to undermine the stability of the system as a whole just to make it build.
We use the official CMake macros to detect the Python version. On Linux systems it will detect /usr/bin/python which is only a symbolic link to the actual Python executable and on most systems still points to Py2.
After running CMake configure the first time you must manually change it to /usr/bin/python3 and remove the other CMake Python variables that may point to the wrong include/library path. When re-running CMake configure it should set automatically the correct values for the other Python variables.
I suspect cmake has not been set up properly or is assuming a Debian* specific configuration which does not work on other distros.
As said we use the official CMake macros and it depends on them how platform-independent they are implemented.
User avatar
freman
Posts: 775
Joined: Tue Nov 27, 2018 10:30 pm

Re: toolmanager bombed out , all tools gone.

Postby freman » Tue Oct 08, 2019 1:49 pm

Then try the combination Py3/Qt4.
That is what I was doing when I got the mixed py versions. This is a bit long, so I'll recap the problems.

1. py2 build is broken for path. That is my primary interest in FC at this time.
2. if I try -DPYTHON_EXECUTABLE=/usr/bin/python3 it gets most of it right but throws the mix of py2.7 covered above.
3. Until I can fix "qt5uitools" issue py3/qt5 does not make it through cmake. I suspect when it does I will similarly hit no.2 again.
User avatar
freman
Posts: 775
Joined: Tue Nov 27, 2018 10:30 pm

Re: toolmanager bombed out , all tools gone.

Postby freman » Tue Oct 08, 2019 2:00 pm

On Linux systems it will detect /usr/bin/python which is only a symbolic link to the actual Python executable and on most systems still points to Py2.
It seems that you have an override in the form of the cmake option. This work in all places but pyside.

Is pyside using a different sniffing method?
Could this be short circuited to allow previous correct detection to persist?
wmayer
Site Admin
Posts: 14999
Joined: Thu Feb 19, 2009 10:32 am

Re: toolmanager bombed out , all tools gone.

Postby wmayer » Tue Oct 08, 2019 2:04 pm

2. if I try -DPYTHON_EXECUTABLE=/usr/bin/python3 it gets most of it right but throws the mix of py2.7 covered above.
CMake creates some variables for Shiboken and PySide and there you have to make sure it doesn't point to any Py2 stuff.

It's best to use the CMake GUI and then go through all variables that is about Python, shiboken or PySide. I remember it was a PITA when I first built FreeCAD with Py3 under Ubuntu until I got it working.
User avatar
freman
Posts: 775
Joined: Tue Nov 27, 2018 10:30 pm

Re: toolmanager bombed out , all tools gone.

Postby freman » Tue Oct 08, 2019 2:42 pm

Thanks, I'd already looked at that. All the python variables look good, all 3.7
Shiboken_DIR and PySide_DIR are pointing to /lib64/cmake/Shiboken-1.2.4 etc

/lib64 is a symlink to /usr/lib64 where both the python2.7 adn python 3.7 structure both reside.

Shiboken seems to be the culprit:

Code: Select all

if (NOT PYTHON_SUFFIX)
    message(STATUS "Using default python: -python2.7")
    SET(PYTHON_SUFFIX -python2.7)
endif()
include(/usr/lib64/cmake/Shiboken-1.2.4/ShibokenConfig${PYTHON_SUFFIX}.cmake)
PySide:

Code: Select all

if (NOT PYTHON_BASENAME)
    message(STATUS "Using default python: .cpython-37m-x86_64-linux-gnu")
    SET(PYTHON_BASENAME .cpython-37m-x86_64-linux-gnu)
endif()
include(/usr/lib64/cmake/PySide-1.2.4/PySideConfig${PYTHON_BASENAME}.cmake)
It is not clear at first view where these get set or why they use a different variable name . Presumably these could be set as -D options to force the right result.

Code: Select all

cmake -DPYTHON_EXECUTABLE=/usr/bin/python3 -DPYTHON_SUFFIX=-python2.7  ../freecad-source
...
-- libshiboken built for Release
-- Using default python: .cpython-37m-x86_64-linux-gnu
Seems to fix the cmake , I'll build and see what happens. ;)

WTF? I typed in 2.7 and forgot to change it and it says it's using .cpython-37m-x86_64-linux-gnu :?
vocx
Posts: 1912
Joined: Thu Oct 18, 2018 9:18 pm

Re: toolmanager bombed out , all tools gone.

Postby vocx » Tue Oct 08, 2019 3:54 pm

freman wrote:
Tue Oct 08, 2019 10:19 am
CompileOnUnix:
cmake -DPYTHON_EXECUTABLE=/usr/bin/python3 ../freecad-source
default python is indeed py2 on this machine, it seems that the cmake option is not ensuring py3 in all places. Is this an oversight?

Can anyone explain what this is all about? I built coin, pivy, occt and opencamlib locally, do I need to rebuild any / all of those if I use py3 build of FreeCAD?
CompileOnUnix
If that doesn't work, you may have to define additional variables pointing to the desired Python libraries and include directories:

Code: Select all

cmake -DPYTHON_EXECUTABLE=/usr/bin/python3.6 \
    -DPYTHON_INCLUDE_DIR=/usr/include/python3.6m \
    -DPYTHON_LIBRARY=/usr/lib/x86_64-linux-gnu/libpython3.6m.so \
    -DPYTHON_PACKAGES_PATH=/usr/lib/python3.6/site-packages/ \
    ../freecad-source
I don't understand why you build Coin, Pivy, OCCT and all that; it sounds complicated. If you cannot solve this issue of using Python 3, I have to wonder why you even did that, instead of just using Fedora packages, or a Flatpack, or the AppImage. If you already compiled all those dependencies, maybe you also need to compile Qt5 and PySide2?

This user seems to be using Fedora with a package from their repositories.
fillet wrote:
Thu Jun 06, 2019 10:33 pm
I'm using Fedora 30.
I try the last AppImage using OCC7.3 (FreeCAD_0.18-16117-Linux-Conda_Py3Qt5_glibc2.12-x86_64.AppImage). I open Freecad. When I try to open a freecad file, it crashes.
So I'm using Freecad version not from official website appimage, but from official Fedora repository. It is not the last OCC version, but it works !
I hope you will solve your AppImage problem for Fedora...
freman wrote:
Tue Oct 08, 2019 12:02 pm
It seems those working on the code using linux seem to be focused on Ubuntu and there is little testing beyond that. All snapshot builds depend on PPA Ubuntu specific installation.

Package naming is one thing which changes with different distros and I am still trying to find out what package is ( apparently ) missing to build FC using qt5 on Fedora 29. There is no help on that in the guide.
AppImages are snapshot builds. They are supposed to run in all Linux distributions. That's a much easier way of using the recommended Py3/Qt5 combination rather than compiling or waiting for a package from a repository.

Simply put, Fedora packagers need to do a better job of maintaining their repositories. If I was using Fedora I would have provided better compilation instructions for it in CompileOnUnix, but since I'm using the superior Linux distribution, Ubuntu, I can only focus on what I have at hand.

See this thread from hobbes. It seems Fedora didn't have PySide2 until very recently. Maybe this is the reason you cannot compile against Qt5?
hobbes1069 wrote:
Tue Sep 03, 2019 2:25 pm
I'm working on moving freecad on Fedora from Qt4 to Qt5 now that I have pyside2 packaged for Fedora but I'm running into what looks like an unrelated build error:

https://forum.freecadweb.org/viewtopic.php?f=4&t=38982

Coin 4 is not currently packaged for Fedora but I did find a 4.0.0 source on Coin3D's bitbucket page.

FreeCAD is using Python 3 at this point on Fedora 30 and up (I didn't change it on Fedora 29 IIRC):
...
Also see issue #4132, Patch fedora.spec to get FreeCAD packages building again. Is COPR supposed to be a repository for daily builds? Maybe this would also work for you, freecad/nightly.
User avatar
freman
Posts: 775
Joined: Tue Nov 27, 2018 10:30 pm

Re: toolmanager bombed out , all tools gone.

Postby freman » Tue Oct 08, 2019 4:17 pm

I don't understand why you build Coin, Pivy, OCCT and all that; it sounds complicated. If you cannot solve this issue of using Python 3, I have to wonder why you even did that, instead of just using Fedora packages
This user seems to be using Fedora with a package from their repositories.
Maybe if you read a bit further you would realise.
fillet wrote:
I'm using Fedora 30.
Maybe is should be using the unstable Rawhide version of Fedora, to add python 3.8 and a few more sources of problems to my unstable FreeCAD.

I may be masocistic enough to perceiver with FreeCAD, but I'm not out but build things which are already in the repo. I built things which were not available or where not recent enough versions. Yes it was a PITA.
but since I'm using the superior Linux distribution, Ubuntu, I can only focus on what I have at hand.
LOL. I'm sure you can get some MacOS or Windows users to have nice flame war with , I'm not biting.
It seems Fedora didn't have PySide2 until very recently.
Funny , I don't seem to find any mention of PySide2 on your guide. What is it called on your superior distro?

Does PySide2 provide qt5uitools which was preventing qt5 build from getting through cmake?
vocx
Posts: 1912
Joined: Thu Oct 18, 2018 9:18 pm

Re: toolmanager bombed out , all tools gone.

Postby vocx » Tue Oct 08, 2019 4:29 pm

freman wrote:
Tue Oct 08, 2019 4:17 pm
...
Maybe if you read a bit further you would realise.
fillet wrote:
I'm using Fedora 30.
This means absolutely nothing to me. Is this stable, unstable, new, old? What are you trying to say?
Funny , I don't seem to find any mention of PySide2 on your guide. What is it called on your superior distro?

Does PySide2 provide qt5uitools which was preventing qt5 build from getting through cmake?
PySide is the official Qt binding for Python. PySide is for Qt4, while PySide2 is for Qt5. If you have Qt5 you should have PySide2, and all the tools needed for graphical interfaces. But these tools are all in different packages. In the CompileOnUnix package, in the Debian/Ubuntu section, the required packages are listed. As I said, it's unfortunate that Fedora and Suse users don't have good, up to date, packages, but that's a problem of the maintainers of those systems, not of FreeCAD.