Porting to python3

Here's the place for discussion related to coding in FreeCAD, C++ or Python. Design, interfaces and structures.
triplus
Posts: 8757
Joined: Mon Dec 12, 2011 4:45 pm

Re: Porting to python3

Postby triplus » Thu Jul 12, 2018 12:33 pm

looo wrote:
Thu Jul 12, 2018 12:08 pm
Maybe the system libTKMath is used instead of the bundled. Isn't libTKMath a occt library?Maybe I have to reanable this line:
https://github.com/looooo/freecad_conda ... /AppRun#L4
That is the exact problem i want to avoid. When reusing PPA/Ubuntu/Debian packages somebody else likely took care of that. The downside being you can't just use the newest libraries with ease. Creating AppImage from scratch gives you that possibility. But with an app like FreeCAD and all its dependencies that comes down to a whole lot of investing effort.
Are you using occt7.1 on this system?
This system is used for PPA AppImage testing purposes and it uses version 7.2 (from PPA).
Theoretically it should work, as all packages are build with centos6 which was released in 2014. Previous tests with conda-environments on ubuntu 14.04 also worked...
Good to hear that. Note that i personally would love to see it work. You invest a lot of effort in packaging therefore if that could be turned into an AppImage. That for sure would be good.
edit: example: I installed command-panel and FreeCAD crashes when switching to sketcher-workbench
Could be. Will take a look tomorrow. This things happen sometimes. I test on Py3/Qt5 version available on Ubuntu 16.04. But newer Py3/Qt5 versions sometimes introduce different behavior. Will likely need to add an option with latest Py3/Qt5 to the test matrix. But i guess i can't remove the older options from the matrix as end users still use them too. ;)
looo
Posts: 2917
Joined: Mon Nov 11, 2013 5:29 pm

Re: Porting to python3

Postby looo » Thu Jul 12, 2018 12:48 pm

triplus wrote:That is the exact problem i want to avoid.
Can you explain?
triplus wrote:When reusing PPA/Ubuntu/Debian packages somebody else likely took care of that.
if we suggest to use appimages on non debian based system I guess we cannot guarantee for system-libraries having the same name as bundled libraries. Therefor I guess setting some environment variables is absolutely necessary.
triplus wrote:This system is used for PPA AppImage testing purposes and it uses version 7.2 (from PPA).
Currently I am also using occt7.2. But allthough they should be binary compatible they are not. I guess some dependencies of occt are not compatible...
triplus wrote:Good to hear that. Note that i personally would love to see it work. You invest a lot of effort in packaging therefore if that could be turned into an AppImage. That for sure would be good.
Yes, somehow people don't like package manager. Plug and play is still the preferred way.
please help with my conda-packaging efforts: https://liberapay.com/looooo/
triplus
Posts: 8757
Joined: Mon Dec 12, 2011 4:45 pm

Re: Porting to python3

Postby triplus » Thu Jul 12, 2018 1:07 pm

looo wrote:
Thu Jul 12, 2018 12:48 pm
Can you explain?
if we suggest to use appimages on non debian based system I guess we cannot guarantee for system-libraries having the same name as bundled libraries. Therefor I guess setting some environment variables is absolutely necessary.
Travis produced around 400 daily FreeCAD Py2/Qt4 AppImage builds until now. We only had to fix a few minor reported glitches. And in the end that works for me. If we want more control, newer libraries ... investment of the effort needed to achieve and sustain such luxury doesn't work for me ATM. ;) For that you need to be prepared to invest effort in this area on a daily basis. Basically you need to love packaging more then other things. Or it won't work. I just don't love packaging that much ATM.
triplus
Posts: 8757
Joined: Mon Dec 12, 2011 4:45 pm

Re: Porting to python3

Postby triplus » Thu Jul 12, 2018 1:50 pm

Anyway as for the short version. The goal is to produce (daily) FreeCAD Py3 AppImage. And ATM i don't care if it is PPA/Conda based or if we do them both. And if in any workbench a problem is detected or not. We can addresses that after. I am familiar with the whole procedure on how to create AppImage from the PPA. Can you post instructions on how the whole procedure looks like if initiated from stock Ubuntu 14.04/16.04. Will test that tomorrow and see how it goes.
looo
Posts: 2917
Joined: Mon Nov 11, 2013 5:29 pm

Re: Porting to python3

Postby looo » Thu Jul 12, 2018 2:10 pm

triplus wrote:Travis produced around 400 daily FreeCAD Py2/Qt4 AppImage builds until now. We only had to fix a few minor reported glitches.
I have no idea how the current appimage-process works, but it looks quite difficult. (I think the current appimages also uses some environment-variables to find the right libraries...)
Also there is a security-bug reported regarding the way how we are doing appimages currently. https://github.com/AppImage/AppImages/issues/197
triplus wrote:For that you need to be prepared to invest effort in this area on a daily basis. Basically you need to love packaging more then other things. Or it won't work. I just don't love packaging that much ATM.
I don't think there is too much difference between an appimage based on ubuntu and one based on conda-packages. The idea of AppImage is to isolate the libraries from the system and only use the most basic ones. The same is true for conda. So bundeling a conda environment with appimage is maybe not the worst idea.
For sure ubuntu packages have a better quality/stability. But the step from building some sources locally to creating a ubuntu/debian package is huge. This isn't true for conda.
triplus wrote:Basically you need to love packaging more then other things. Or it won't work. I just don't love packaging that much ATM.
I gave up ubuntu packaging because it was too time-intensive. With conda I have much more success. Eg.: Today I managed to build external smesh with netgen on osx although I never touched a mac. Also freecad on osx (but without netgen-support) is available since some weeks. But for sure it's time-intensive too.
please help with my conda-packaging efforts: https://liberapay.com/looooo/
looo
Posts: 2917
Joined: Mon Nov 11, 2013 5:29 pm

Re: Porting to python3

Postby looo » Thu Jul 12, 2018 8:57 pm

@triplus: I added the library-path variable and uploaded a new appimage. It would be nice if you can try again.
please help with my conda-packaging efforts: https://liberapay.com/looooo/
looo
Posts: 2917
Joined: Mon Nov 11, 2013 5:29 pm

Re: Porting to python3

Postby looo » Fri Jul 13, 2018 7:09 am

triplus wrote:Can you post instructions on how the whole procedure looks like if initiated from stock Ubuntu 14.04/16.04. Will test that tomorrow and see how it goes.
It's easy:
- Install miniconda.
- add conda-forge channel: conda config --add channels conda-forge
- activate the root environment
- git clone https://github.com/looooo/freecad_conda_appimage
- cd freecad_conda_appimage
- bash create_appimage.sh (downloads and installs all necessary libraries from conda-forge + create appimage)

You don't have to do this in a clean system, as nothing from system libraries is bundled, and no package is build at all. Building FreeCAD itself is done in a defined docker-environment on circle-ci. So creating the appimage only gathers all the sources.
please help with my conda-packaging efforts: https://liberapay.com/looooo/
looo
Posts: 2917
Joined: Mon Nov 11, 2013 5:29 pm

Re: Porting to python3

Postby looo » Fri Jul 13, 2018 2:01 pm

I tried on another machine and can reproduce the problem with libtkmath. At least for me it seems there is missing tbb in the appimage.
please help with my conda-packaging efforts: https://liberapay.com/looooo/
triplus
Posts: 8757
Joined: Mon Dec 12, 2011 4:45 pm

Re: Porting to python3

Postby triplus » Fri Jul 13, 2018 9:16 pm

looo wrote:
Thu Jul 12, 2018 2:10 pm
I have no idea how the current appimage-process works, but it looks quite difficult. (I think the current appimages also uses some environment-variables to find the right libraries...)
Currently we reuse the work done by PPA/Ubuntu/Debian package maintainers. Not just FreeCAD/OCC stuff but everything FreeCAD depends on. And pack that in an AppImage.
I don't think there is too much difference between an appimage based on ubuntu and one based on conda-packages. The idea of AppImage is to isolate the libraries from the system and only use the most basic ones. The same is true for conda. So bundeling a conda environment with appimage is maybe not the worst idea.
For sure ubuntu packages have a better quality/stability. But the step from building some sources locally to creating a ubuntu/debian package is huge. This isn't true for conda.
There is not much difference. In addition you don't have to use PPA/Ubuntu/Debian or Conda approach. You can create AppImages directly from scratch. Using the PPA/Debian/Ubuntu approach for Py2/Qt4 builds takes the least amount of effort and it enabled us to get results. That is why it is used ATM.
Also there is a security-bug reported regarding the way how we are doing appimages currently. https://github.com/AppImage/AppImages/issues/197
Sure. Security can always be improved. If they plan to add better support for checking things like GPG signature and preventing possible MIT attacks. That all sounds good to me.
I gave up ubuntu packaging because it was too time-intensive. With conda I have much more success. Eg.: Today I managed to build external smesh with netgen on osx although I never touched a mac. Also freecad on osx (but without netgen-support) is available since some weeks. But for sure it's time-intensive too.
Likely it doesn't differ all that much. It's just with Conda you have more control and can just do something that works (for you). And that is that. If you are good in .deb packaging you can use the same approach on PPA. And you can use the same approach when creating AppImage. If you want to make one from scratch. But we said the plan is two weeks for producing a Py3 AppImage and not two years. And therefore this is something you or somebody else can explore in the future. ;) I am still on the two weeks plan.
It's easy:
I'll take a look over the weekend.
looo
Posts: 2917
Joined: Mon Nov 11, 2013 5:29 pm

Re: Porting to python3

Postby looo » Sat Jul 14, 2018 7:12 pm

I have updated the appimage again (tbb included). For newer distros (ubuntu18.04) there is a problem with the menus. Should be solved with newer qt5 version (but I can't update this now).

So this is all for now. Now I will begin with my "aggresive" 2-weeks-plan. (bike-holyday-trip :D )

I wish you luck with ubuntu-python3.
please help with my conda-packaging efforts: https://liberapay.com/looooo/