appimage

Here's the place for discussion related to coding in FreeCAD, C++ or Python. Design, interfaces and structures.
triplus
Posts: 8690
Joined: Mon Dec 12, 2011 4:45 pm

Re: appimage

Postby triplus » Sat Jan 26, 2019 12:52 am

Kunda1 wrote:
Sat Jan 26, 2019 12:14 am
I see. Thanks for the correction and clarification. :wink:
You're welcome.
Lets make a list then of what is needed... Maybe it can be posted to social media to see if we get any backers?
More knowledge, hardware resources and automation. ;)
User avatar
Kunda1
Posts: 5576
Joined: Thu Jan 05, 2017 9:03 pm

Re: appimage

Postby Kunda1 » Sat Jan 26, 2019 1:15 am

triplus wrote:
Sat Jan 26, 2019 12:52 am
More knowledge, hardware resources and automation. ;)
These are the things I'm trying to ascertain.
Automation
In regards to automation: Currently FC does have a freecad.io that has the ability to automate tasks. I'm running a script that updates AppImage link in the wiki. I started a page Backend Adminstration that will document backend tasks. So by all means we have some automation ability. Also there is an open ticket about buildbot automation in issue #3259. It requires some understanding with virtual machines etc...

Hardware Resources
What does this look like? examples please

Knowledge
The eternal search
Image
Want to contribute back to FC? Checkout:
#lowhangingfruit | Use the Source, Luke. | How to Help FreeCAD | How to report FC bugs and features
User avatar
Kunda1
Posts: 5576
Joined: Thu Jan 05, 2017 9:03 pm

Re: appimage

Postby Kunda1 » Sat Jan 26, 2019 1:24 am

Kunda1 wrote:
Sat Jan 26, 2019 12:14 am
triplus wrote:
Sat Jan 26, 2019 12:06 am
Nop. Both AppImages are created and deployed on a daily basis. Therefore whenever @looo-ci updates the Conda based packages (currently once a week), that gets picked up by AppImage in less then 24h. As for the missing file, that therefore should fix itself tomorrow.
I see. Thanks for the correction and clarification. :wink:
Nevertheless, I've added issue #3797 to track this edge-case.
Want to contribute back to FC? Checkout:
#lowhangingfruit | Use the Source, Luke. | How to Help FreeCAD | How to report FC bugs and features
looo
Posts: 2825
Joined: Mon Nov 11, 2013 5:29 pm

Re: appimage

Postby looo » Sat Jan 26, 2019 8:16 am

regarding daily conda-packages:

Currently I start 3 full builds (win, linux osx) (2h of build time) once a week. If we want to do this daily the current workflow is not good. I guess some caching should be added to make the builds faster.

One way to do this is using docker. We simple base the docker-file on condaforge/linux-anvil and build FreeCAD with conda. We upload the resulting docker-image and use it as a base for the appimage. (additional a entrypoint for a script is necessary, but this should be possible with some research)

Now for the daily builds we simple have to:
1. update the docker image (docker pull freecad/conda-daily) and cache this image somehow
2. create a container (docker run freecad/conda-daily ...)
3. this docker container than
- pulls github sources of freecad
- builds freecad with conda-build and the --dirty flag (uses the old build and is much faster than a full build)
- installs the created freecad-package in the appimage-environment (maybe optional)
- creates the appimage (maybe optional)
- returns the conda-package (and maybe the appimage)

This way everyone should be able to create a appimages and conda-packages of FreeCAD. And we can build FreeCAD in a fast(er) way on travis. But we still base the resulting package/appimage on a rolling release and therefor can influence included dependencies.


If we aim for cross-plattform daily builds this idea is not so good, as docker only works for linux. But I don't know if there is a way to create bundles the same for all plattforms...
User avatar
sgrogan
Posts: 5344
Joined: Wed Oct 22, 2014 5:02 pm

Re: appimage

Postby sgrogan » Sat Jan 26, 2019 4:06 pm

looo wrote:
Sat Jan 26, 2019 8:16 am
If we aim for cross-plattform daily builds this idea is not so good, as docker only works for linux. But I don't know if there is a way to create bundles the same for all plattforms...
Maybe as a base we could just cache the miniconda installation?

The way I do this locally on win is to; install miniconda and create a FreeCAD env in conda. Maybe this could be cached?
I then use CMake to make a build over the conda env in the "Library" directory. I then run the Install project to get all the FreeCAD stuff.

I then copy the dependencies from the conda env to the correct place in the install directory structure . Then on win 7zip it up.
On Win this amounts to a white list handling of the dependencies.

If I understand, Peter did similar for the OSX/conda .dmg. IIUC he uses a tool to copy the dependencies, so it should be easier to maintain.

I think triplus copies the environment and deletes stuff (black list) for Linux/AppImage.

On Win I don't attempt to keep up with conda's rolling release of the dependencies. I re-use a known stable env. When I have time I create a new env, work out any kinks, then use this new env until the process is repeated.

looo has mentioned caching the sources, I wonder if we will run into storage capacity limits? We only need the history to get the version info, but this is another case of worms.

I don't think hardware is limiting us. We need the knowledge (and the time to acquire it), and the time to automate this knowledge.

We're only able to have the discussion because of looo's conda work, so @looo thanks for this.
triplus
Posts: 8690
Joined: Mon Dec 12, 2011 4:45 pm

Re: appimage

Postby triplus » Sat Jan 26, 2019 11:48 pm

I am guessing eventually we would like to have Travis CI up and running for all 3 platforms? If we ever manage to achieve that i wonder if we could use Conda for getting the dependencies and setting up build environment, opposed to using Travis + Homebrew/LibPack/deb/PPA ... That would i guess enable us to deploy packages for all 3 platforms on "each merge" and to bump the FreeCAD package version on Conda in i guess fully automated way. Basically what macOS team has already been doing to some extent.

But then again maintaining Homebrew/LibPack support still would be desirable? As Conda isn't the fastest moving package manager, when it comes to packages such as Qt.

P.S. For Linux, we could likely explore and provide FreeCAD 0.19 daily builds based on the Ubuntu 18.04 daily PPA, as an additional option. Such AppImage wouldn't work with older Linux distributions, but it likely would with the latest ones.
rentlau_64
Posts: 181
Joined: Tue Oct 21, 2014 9:47 am
Location: Pau (France)

Re: appimage

Postby rentlau_64 » Wed Jan 30, 2019 5:35 pm

guys
thank you for your comments
my pyside2 is installed in my anaconda env
l will try to see if i have another one in my system
and try to bypass the one in anaconda.
will keep you inform.
thank you
I just use :

Code: Select all

pip uninstall PySide2
Uninstalling PySide2-5.12.0:
  Would remove:
    /home/laurent/.local/bin/pyside2-lupdate
    /home/laurent/.local/bin/pyside2-rcc
    /home/laurent/.local/bin/pyside2-uic
    /home/laurent/.local/lib/python3.6/site-packages/PySide2-5.12.0.dist-info/*
    /home/laurent/.local/lib/python3.6/site-packages/PySide2/*
    /home/laurent/.local/lib/python3.6/site-packages/pyside2uic/*
Proceed (y/n)? y
  Successfully uninstalled PySide2-5.12.0
And I have no more issue launching appimage QT5 py3: FreeCAD_0.18.15671_Conda_Py3Qt5_glibc2.12-x86_64.AppImage

I will now try to change WorkFeature WB and Macro to be python3 compatible
Rentlau
triplus
Posts: 8690
Joined: Mon Dec 12, 2011 4:45 pm

Re: appimage

Postby triplus » Wed Feb 06, 2019 6:51 pm

OK therefore there is an issue with libexpat library? Not just AppImage related? And we suspect this is a general issue with newest version of libexpat? Can somebody having this issues point me in the right direction. To do some tests.

As for Travis to GitHub AppImage deployment. I checked the logs for the past few days and deployment worked more or less reliably. I don't feel any improvement are needed ATM.
fosselius wrote:
Sun Jan 20, 2019 8:32 pm
The app img do not show menu text (drop down menu are blank) and the spacing in the top menu is wrong..
On the freecad-daily, whats up with the icons?
AFAIK the AppImage issue should go away once Qt5 on Conda gets upgraded. As for the FreeCAD daily. Disable the built in icon theme and the issue should go away:

Code: Select all

p = FreeCAD.ParamGet("User parameter:BaseApp/Preferences/Bitmaps/Theme")
p.SetString("Name", "None")
https://forum.freecadweb.org/viewtopic. ... 10#p276881

As for the Python libraries installed through pip, PIPTools and in general libraries being available on user system (environment). And what AppImage should see by default and what it should not. This is a process we likely need to understand better first. Before we make changes. If some library is in clash, therefore it can get removed. The same issue was reported a few days back, regarding FreeCAD daily, from PPA, and a clash happened with pip installed PySide2 (official wheel).
triplus
Posts: 8690
Joined: Mon Dec 12, 2011 4:45 pm

Re: appimage

Postby triplus » Sun Feb 10, 2019 6:13 pm

I noticed Conda based AppImage now uses native style on Ubuntu 16.04. Nice!
fosselius wrote:
Sun Jan 20, 2019 8:32 pm
The app img do not show menu text (drop down menu are blank) and the spacing in the top menu is wrong..
Please test again with latest Conda AppImage.
triplus
Posts: 8690
Joined: Mon Dec 12, 2011 4:45 pm

Re: appimage

Postby triplus » Sun Feb 10, 2019 10:58 pm

@looo

If i remember correctly you added external Ship and Plot modules to Conda based AppImage? I did some workbench switching today. Mainly to determine general icon themes support state in default set of workbenches and noticed a small issue in Plot module (Conda AppImage):

Code: Select all

name 'FreeCAD' is not defined
usr/lib/python3.6/site-packages/freecad/plot/init_gui.py", line 62, in Initialize
    FreeCAD.Console.PrintMessage(msg + '\n')