Porting to python3

Here's the place for discussion related to coding in FreeCAD, C++ or Python. Design, interfaces and structures.
triplus
Posts: 8342
Joined: Mon Dec 12, 2011 4:45 pm

Re: Porting to python3

Postby triplus » Sat Jul 21, 2018 11:08 am

Hi @looo. I don't have much time ATM (likely until Monday). If you will take a look here you might notice something:

https://github.com/FreeCAD/FreeCAD/releases

We are at two successful builds now (and counting). Therefore i guess i am not in aggressive mode anymore. The goal was reached. ;)

P.S. I was wondering on how hard would it be to for example upgrade Qt/PySide to lets say 5.11? Do you have to do all that manually yourself for FreeCAD purposes or can you reuse such builds from other Conda packagers? I am guessing regardless of using Conda that packaging and upgrading something like Qt/PySide is still PITA?
looo
Posts: 2481
Joined: Mon Nov 11, 2013 5:29 pm

Re: Porting to python3

Postby looo » Sat Jul 21, 2018 11:37 am

triplus wrote:We are at two successful builds now (and counting). Therefore i guess i am not in aggressive mode anymore. The goal was reached
can you give some insights on how you build the appimage?
triplus wrote:P.S. I was wondering on how hard would it be to for example upgrade Qt/PySide to lets say 5.11? Do you have to do all that manually yourself for FreeCAD purposes or can you reuse such builds from other Conda packagers? I ma guessing regardless of using Conda packaging something like that is still PITA?
Yes I already looked how to get qt updated, but it's not that easy.
First there is a qt5.9 package [1] which should work for freecad. But this qt5.9 package is made with a different toolchain (mainly this toolchain is based on gcc7.2 which is provided by the anaconda-defaults channel.) Conda-forge (the community-packages) are still build with gcc-4.8.* and all dependencies of freecad uses this toolchain. I tried already to build pyside with qt5.9[2] from anaconda but mixing different compilers is not a good idea. So after some discussion I think there are 2 options:

1. update the toolchain to newer compilers:
this is the goal of anaconda (as it is a big benefit if anaconda and conda-forge is compatible). But as I said this is a huge work. Many packages have to be rebuild. Currently they (conda-forge) prepare to get everything ready to make the switch. I guess conda-forge has a good overview of things that have to happen to reach the goal. There is a bot which updates all the packages to use latest conda-build-commands and pinning. Once this is done I guess conda-forge will try to make the change to new compilers... Once that is done and conda-forge and anaconda use the same compilers we can think about updating to qt5.9 or qt5.12 (which will be released in November). For qt5.12 I think anaconda will provide a package, so we don't have to do it on our own.

2. update the conda-forge qt package:
Someone already tried this, but qt is known to be very difficult, especially as a conda-package [3]. I also tried to compile qt5.11 (without webkit) locally but I gave up pretty soon.

So in my eyes it's best to stay with qt5.6 for now and concentrate on the other tasks until a conda-forge compatible qt5.12 is available. I guess this will be the case at the end of 2018.


[1] https://anaconda.org/anaconda/qt
[2] https://github.com/conda-forge/pyside2- ... ck/pull/19
[3] https://github.com/conda-forge/qt-feedstock/pull/66
triplus
Posts: 8342
Joined: Mon Dec 12, 2011 4:45 pm

Re: Porting to python3

Postby triplus » Sat Jul 21, 2018 11:50 am

looo wrote:
Sat Jul 21, 2018 11:37 am
can you give some insights on how you build the appimage?
This is where both AppImages from FreeCAD releases page are built:

https://github.com/FreeCAD/FreeCAD-AppImage
Yes I already looked how to get qt updated, but it's not that easy.
Good luck. ;)
So in my eyes it's best to stay with qt5.6 for now and concentrate on the other tasks until a conda-forge compatible qt5.12 is available. I guess this will be the case at the end of 2018.
+1

We have Py2/Qt4 AppImage and Py3/Qt5 AppImage now. Therefore both PPA and Conda effort is being reused and should propagate to the end users. In addition enabling more Py3 testing and feedback was the big goal. And that should be possible now.
looo
Posts: 2481
Joined: Mon Nov 11, 2013 5:29 pm

Re: Porting to python3

Postby looo » Sat Jul 21, 2018 12:57 pm

good work! As currently this recipe won't update to the latest packages I guess it's best to send a PR once the conda-forge package is updated. With my work on circle-ci it would be possible to build freecad for conda and create the appimage afterwards. But this will need some more time until it gets ready...
triplus wrote:In addition enabling more Py3 testing and feedback was the big goal.
Yes this is the main benefit. I hope some people can try to work on projects with python3 and report problems.
looo
Posts: 2481
Joined: Mon Nov 11, 2013 5:29 pm

Re: Porting to python3

Postby looo » Sun Jul 22, 2018 2:49 pm

I don't know why my approach doesn't work for circle-ci. All the conda-stuff works but the appimage can't be created:

Code: Select all

+ ARCH=x86_64
+ squashfs-root/AppRun AppDir
appimagetool, continuous build (commit 504b6d2), build 1788 built on 2018-07-18 14:48:47 UTC
WARNING: appstreamcli command is missing, please install it if you want to use AppStream metadata
Desktop file not found, aborting
https://circleci.com/gh/looooo/FreeCAD/ ... build-link
triplus
Posts: 8342
Joined: Mon Dec 12, 2011 4:45 pm

Re: Porting to python3

Postby triplus » Thu Jul 26, 2018 4:30 pm

Hi @looo

Likely some path related issue is involved. If the desktop file is provided but can't be found. On Travis i added:

Code: Select all

$TRAVIS_BUILD_DIR/conda/AppDir
Therefore look for similar environment variables (if available by CircleCI). But then again why? Daily Conda based AppImage gets created and deployed already. And it doesn't need any additional effort invested to reach this goal. Things like investigating on how to provide correct version information and on what would be needed to update FreeCAD Conda package on a more daily alike basis. That should be much higher on the priority list in my opinion.
User avatar
kkremitzki
Posts: 1526
Joined: Thu Mar 03, 2016 9:52 pm
Location: Texas

Re: Porting to python3

Postby kkremitzki » Thu Jul 26, 2018 6:25 pm

There's been an update on the pyside2 packaging bug in Debian, it looks like Raphaël Hertzog has pretty much finished the work: https://bugs.debian.org/cgi-bin/bugrepo ... =877871#63
Like my FreeCAD work? I'd appreciate any level of support via Patreon, Liberapay, or PayPal! Read more about what I do at my blog.
User avatar
NormandC
Posts: 18534
Joined: Sat Feb 06, 2010 9:52 pm
Location: Québec, Canada

Re: Porting to python3

Postby NormandC » Thu Jul 26, 2018 7:35 pm

Thanks for the update, some interesting info in Raphaël's post:

Raphaël Hertzog wrote:The initial packaging of pyside has been funded by a customer who is using freecad and wanted to keep the package in Debian. We will continue to help for as long as the customer is willing to pay our work but we definitely want the package to be under the team's umbrella so that it survives our efforts.
User avatar
kkremitzki
Posts: 1526
Joined: Thu Mar 03, 2016 9:52 pm
Location: Texas

Re: Porting to python3

Postby kkremitzki » Thu Jul 26, 2018 10:19 pm

NormandC wrote:
Thu Jul 26, 2018 7:35 pm
Thanks for the update, some interesting info in Raphaël's post:

Raphaël Hertzog wrote:The initial packaging of pyside has been funded by a customer who is using freecad and wanted to keep the package in Debian. We will continue to help for as long as the customer is willing to pay our work but we definitely want the package to be under the team's umbrella so that it survives our efforts.
Indeed, hopefully it's OK to say that again we have Aleph Objects to thank and that if anyone reading this is considering a 3D printer purchase in the US, to consider a LulzBot! :)
Like my FreeCAD work? I'd appreciate any level of support via Patreon, Liberapay, or PayPal! Read more about what I do at my blog.
triplus
Posts: 8342
Joined: Mon Dec 12, 2011 4:45 pm

Re: Porting to python3

Postby triplus » Fri Jul 27, 2018 12:19 pm

Yeah. That is what i was afraid of. Packaging Qt/PySide2 sounds like PITA. True that they provide pip-installable PySide2 wheel snapshots and that part is easy. From the user point of view. But i don't believe we can just reuse that easily for our purposes? Debian/PPA/Conda ... needs that to be repacked and to be more Linux alike packaging? In addition Qt/PySide2 using different build system i guess that only adds to the task being harder. Using something like CMake could hopefully make the task a bit easier. But if i remember correctly that is a problem for them as they support platforms where CMake isn't an option.

Anyway if anybody has more in-dept technical insight in this area one option could be to share opinion in such campaigns:

https://www.phoronix.com/scan.php?page= ... ild-System

Basically what we need is a Qt/PySide2 Debian/PPA/Conda packages and therefore for that task to be achievable in some rather straightforward and standard way. I have a feeling both Qt and PySide2 projects will still need to invest some effort in improving this area in the future.