AppImages of v0.18 release builds

Here's the place for discussion related to coding in FreeCAD, C++ or Python. Design, interfaces and structures.
User avatar
sgrogan
Posts: 5412
Joined: Wed Oct 22, 2014 5:02 pm

Re: AppImages of v0.18 release builds

Postby sgrogan » Tue Apr 09, 2019 10:29 pm

looo wrote:
Tue Apr 09, 2019 9:12 pm
I will continue with providing conda-packages, but stop the bundeling efforts. This is simple too much frustration and was also never my target.
Great!

Thanks looo, I think you started this to facilitate the PY3 migration. I think you have accomplished this. I wouldn't have been able to provide PY3 builds on Win without your work. It grew to AppImages and DMG's. I haven't given up on Conda as an universal package manager, but I understand your frustration.
triplus
Posts: 8710
Joined: Mon Dec 12, 2011 4:45 pm

Re: AppImages of v0.18 release builds

Postby triplus » Tue Apr 09, 2019 11:17 pm

I made some progress today, will respond to discussion once i have a bit more results to share. One thing i do find a bit strange and will need to get used to is all the FreeCAD 0.18 sections emerging:

https://github.com/FreeCAD/FreeCAD/releases

Therefore if some package for some platform will have an issue in it. Ideally all packages would need to get changed? For .1, .2, .3 ... point release sections? I would prefer if there would only be one section for each release, providing only the latest point release packages
looo
Posts: 2898
Joined: Mon Nov 11, 2013 5:29 pm

Re: AppImages of v0.18 release builds

Postby looo » Wed Apr 10, 2019 9:14 am

Kunda1 wrote:
Tue Apr 09, 2019 10:11 pm
bummer
I tested the macos build again and it seems like the same crash (after running tests) also occurs in the conda-environment. So it seems like the problem is not caused by the way of creating the bundle.
Also I did some tests with an older build (commit 2a6655e1768d80e0098d410989ae02c622f2e698 august 2018) and there were no crashes. As this version used the same dependencies as the crashing version I guess the problem must related to some changes to FreeCAD.

regarding the problem of the appimage on arch-linux:
I can't provide any fix for the addon-manager problem with git. But I guess it's still possible to release the bundle. Maybe we find a solution in the 2019 development-cycle.
please help with my conda-packaging efforts: https://liberapay.com/looooo/
User avatar
sgrogan
Posts: 5412
Joined: Wed Oct 22, 2014 5:02 pm

Re: AppImages of v0.18 release builds

Postby sgrogan » Wed Apr 10, 2019 10:14 pm

triplus wrote:
Tue Apr 09, 2019 11:17 pm
I made some progress today, will respond to discussion once i have a bit more results to share. One thing i do find a bit strange and will need to get used to is all the FreeCAD 0.18 sections emerging:
We need to discuss this in a new thread. I support the point release tagging but we need some work to make it more friendly.
triplus
Posts: 8710
Joined: Mon Dec 12, 2011 4:45 pm

Re: AppImages of v0.18 release builds

Postby triplus » Wed Apr 10, 2019 11:47 pm

I made some further progress today and will likely try to put some of that in action tomorrow or the day after tomorrow. I read the rest of the discussion. Therefore documentation from releases page is what should get used and the priority are Py3/Qt5 Conda builds (AppImage and macOS). On Windows side therefore things are almost there too.

@sgrogan

If you, have you ever tried, log into Travis, do you have an access to AppImage related repository settings over there? That is what i was asking for. Maybe i would have it too, haven't tried that yet.
looo wrote:
Tue Apr 09, 2019 1:58 pm
I will use py36 and old toolchain for 0.18.1. Judging from the feedback I think this was the more stable dependency mix.
Sounds good.
looo wrote:
Tue Apr 09, 2019 9:12 pm
I will continue with providing conda-packages, but stop the bundeling efforts.
Not accepted.
sgrogan wrote:
Wed Apr 10, 2019 10:14 pm
We need to discuss this in a new thread. I support the point release tagging but we need some work to make it more friendly.
I would propose to create a continuous and stable tags. If possible put them at the top. Here is where Travis will deploy binaries. Before doing a new release, all binaries, and text, from stable are moved to corresponding section, like FreeCAD 0.18. Point releases are something people packaging for Linux distribution find useful, source code should suffice. Deployment scripts wouldn't have to be adjusted at each release, stable/development AppImages update information would always point to corresponding tag (latest stable or continuous).
User avatar
sgrogan
Posts: 5412
Joined: Wed Oct 22, 2014 5:02 pm

Re: AppImages of v0.18 release builds

Postby sgrogan » Thu Apr 11, 2019 12:14 am

triplus wrote:
Wed Apr 10, 2019 11:47 pm
Not accepted.

sgrogan wrote: ↑
Wed Apr 10, 2019 6:14 pm
We need to discuss this in a new thread. I support the point release tagging but we need some work to make it more friendly.

I would propose to create a continuous and stable tags.
I don't know how to do this. It's either continuous or stable?
triplus
Posts: 8710
Joined: Mon Dec 12, 2011 4:45 pm

Re: AppImages of v0.18 release builds

Postby triplus » Thu Apr 11, 2019 12:38 am

Here is one example:

https://github.com/AppImage/AppImageKit/releases

There is a Continuous build at the top using tag continuous and a Release build underneath it. For release build the tag changes, for Continuous build it's always the same. Beyond that i don't know what the details are.

P.S. For AppImage and macOS builds. Where should we currently deploy them? Development builds to 0.19_pre, what about release builds? Having point release binaries and multiple copies of description text, that is a rather unsettling solution in my opinion. Last point release binaries and a single description, on some stable release tag, that is in my opinion desired.
User avatar
sgrogan
Posts: 5412
Joined: Wed Oct 22, 2014 5:02 pm

Re: AppImages of v0.18 release builds

Postby sgrogan » Thu Apr 11, 2019 3:49 pm

triplus wrote:
Thu Apr 11, 2019 12:38 am
Here is one example:

https://github.com/AppImage/AppImageKit/releases

There is a Continuous build at the top using tag continuous and a Release build underneath it. For release build the tag changes, for Continuous build it's always the same. Beyond that i don't know what the details are.
Thanks for the link. This works because of this ingenious script. https://raw.githubusercontent.com/probo ... /upload.sh
How it works is if a commit to master is not tagged and the build succeeds, the continuous release is deleted, the continuous tag is deleted, a new continuous tag is created at the build commit, and a new continuous release is created, then the artifacts are uploaded. This is equivalent to FreeCAD's 0.19_pre. We don't delete and recreate the release, so the tag and associated auto-deployed source code doesn't track HEAD.

For tagged commits a new release is created and artifacts are uploaded. Old tagged releases are not deleted. This is equivalent to our 0.18, 0.18.1, 0.19, 0.19.1 etc.

For a system like this to work, without making it totally unwieldy, we would need to deploy all artifacts from a single Travis instance. I think this is a good goal, but there is quite some work to achieve this.
triplus wrote:
Thu Apr 11, 2019 12:38 am
P.S. For AppImage and macOS builds. Where should we currently deploy them? Development builds to 0.19_pre, what about release builds? Having point release binaries and multiple copies of description text, that is a rather unsettling solution in my opinion. Last point release binaries and a single description, on some stable release tag, that is in my opinion desired.
I share your concern. I think the the crux is that the point releases are "real" releases. There is not really a difference between 0.18.1 and 0.19, except following the Symantec Versioning rules. We could delete older 0.18.x releases, but we can't delete the tags, they must be immutable. The fact that in the past they weren't is the biggest complaint I've seen from other distro packagers.

TLDR;
For the developer builds it's possible to make the tags "continuous and stable"
For release builds the tags need to be immutable.
Developer builds get deployed to 0.xx_pre
Release builds get deployed to the releases matching the tag they're built against.
triplus
Posts: 8710
Joined: Mon Dec 12, 2011 4:45 pm

Re: AppImages of v0.18 release builds

Postby triplus » Thu Apr 11, 2019 7:38 pm

Therefore "continuous release/tag" gets deleted and recreated each time, before deploying, but that still doesn't explain this to me:
Release build (11) @probonopd probonopd released this on 21 Oct 2018
FreeCAD 0.18 @sgrogan sgrogan released this 28 days ago · 18 commits to releases/FreeCAD-0-18 since this release
Where does the difference come from?
There is not really a difference between 0.18.1 and 0.19, except following the Symantec Versioning rules.
I would say there is? Point release is a release with some additional backports and mostly fixes? People should always be directed to the latest point release binaries, perceiving them as a release binaries? The problem i have now is where exactly to deploy 0.18.1 binaries? As if i deploy them to 0.18.1, then there will still be an older set of binaries located at 0.18. When it comes to AppImage, update information of AppImage, downloaded from 0.18, will point to 0.18, and not to 0.18.1. If there would be a "release" and "continuous". All releases would have update information pointed to "release", and all development oriented AppImages to "continuous", just like the PPA updating works. Latest point release binaries would be moved to 0.18, before we would start deploying next FreCAD release binaries (next year). All existing release/development oriented AppImages out there would therefore always get updated to latest release/development FreeCAD version made available.
Developer builds get deployed to 0.xx_pre
Yes, for now i will deploy to 0.19_pre.
Release builds get deployed to the releases matching the tag they're built against.
Here i would rather deploy point releases to FreeCAD 0.18. And not to have a plethora of point release binaries all over the GitHub releases page. And worry on how to update the whole matrix, once some (packaging) issue gets reported and to fix the issue all binaries from release and point releases would need to get updated. At minimum therefore each point release should have a note and a link, explaining latest point release binaries are located at 0.18?

Today i signed in the Travis. As a member of FreeCAD organization i have access to an overview, could trigger a build from Travis ... For settings i don't have sufficient permissions. People that have access to settings, can set things like environment variables, to determine on where exactly to deploy. On the other hand, there most likely are some secure variables in there and therefore it's a good thing, only a limited set of people has access. In addition i can modify the deploy related variables from the .yml file directly, and that should suffice for completing this task. Today i managed to get most of the work done and i am rather optimistic that tomorrow things should start to move forward a bit.

P.S. One question, are you currently creating Conda Windows builds with a script? If yes, that could likely get automated in the future.
User avatar
sgrogan
Posts: 5412
Joined: Wed Oct 22, 2014 5:02 pm

Re: AppImages of v0.18 release builds

Postby sgrogan » Fri Apr 12, 2019 2:39 pm

triplus wrote:
Thu Apr 11, 2019 7:38 pm
Where does the difference come from?
I don't know how to suppress the
18 commits to releases/FreeCAD-0-18 since this release
message. Maybe it's related to exactly how the tags are created? When creating the tag from the github releases api, a so-called "lightweight tag" is created. I couldn't find the answer in the docs, I need to do some experiments on my github page.
triplus wrote:
Thu Apr 11, 2019 7:38 pm
I would say there is? Point release is a release with some additional backports and mostly fixes?
I mean 0.19 not 0.19_pre.
triplus wrote:
Thu Apr 11, 2019 7:38 pm
The problem i have now is where exactly to deploy 0.18.1 binaries?
Yes, this is the problem. When a release is created on the github release page the source code is auto deployed at the commit the tag points to and the url also uses the tag i.e. https://github.com/FreeCAD/FreeCAD/releases/tag/0.18.1
We could potentially deploy to a static tag, like 0.18, to maintain a static url, and accept that the source code is out of date. When doing this we could create as many point release tags as we want and Distros that build from the tag would be happy (we don't need a github release to create the tag) The issue is some build systems don't use git and they download the source from the release page. We've had many complaints about this in the past. Maybe we could double tag and make this work. We could have a release tag that gets deleted and recreated to keep a static url and point release tag that remains immutable? Distros that don't use git would get the latest point release from the "release" github release? I don't know if "release" is a reserved keyword, I see in the comments in @probono's script that "latest" is.
Maybe for know deploy to 0.18.1 until we can work out the specifics.
triplus wrote:
Thu Apr 11, 2019 7:38 pm
P.S. One question, are you currently creating Conda Windows builds with a script? If yes, that could likely get automated in the future.
I still copy some stuff manually, but it could be scripted. We could also use the new libpack. @saso has done some experiments with this on AppVeyor. I think we can make it work, first step is to build Win on Travis. After that we can work on deploying it.

Thanks for the continued interest. We've come far in the last few years.