AppImage of PR and/or branches

[abdullah]

This is intended mostly for the AppImage experts.

I think it is time for us to empower the community to easily test PRs or branches before merging (so during development).

Currently testers have to clone the branch and compile manually. I am personally indebted to all the nice testers that do this and help increase the quality of the end result and accompany me as a developer in the way. However, I think it is time to improve it. The rationale is:

a) Compiling takes time from the testers. Testers contribute a fixed time to FreeCAD. If you take away compiling time, you get more tests or more features tested. This is time dedicated to interfacing with the Community. Compiling is great but it is not interacting with the community.

b) Many potential testers are not testers because they can not compile or do not have the time to compile. Testers are a fundamental part of FreeCAD. It is the interaction which feeds developers with ideas and encourage them to develop and they somehow bridge pure end-users with the developers.

As it is unthinkable that FC is developed by just one person. It is not reasonable to think that FC will be tested by just one person. We need developers and we need testers. Ok, let's concentrate on testers this time...

I think we have a lot of room for improvement here. More tested functionality means not only better functionality (better adapted to the needs of he community), but also less bugs (so more time to fix the bugs that will inevitably make it way into master, or less time spent in bug fixing by potentially another developer that has to invest some energy in understanding what the other development did). Another advantage can be sharing possible fixes with the community (community acceptance tests).

Practical example, I have a fix for the angle issue, but I am not sure it is what we want. A person that would be delighted to test can not compile the code. This delays things and leads to potentially unwanted solutions.

So to go to the point, that I am being too much verbose: I know nothing about AppImage so maybe it is not even the solution. But I know there is somebody out there in FC who knows, so my question: can we do something about this? (Like generating an AppImage when doing a PR. Or any other alternative you may thing of).

[chrisb]

kkremitzki seems to be active in that area: https://forum.freecadweb.org/viewtopic.php?f=4&t=26106

[abdullah]

kkremitzki seems to be active in that area: https://forum.freecadweb.org/viewtopic.php?f=4&t=26106

Nice to know, I also found this:
https://forum.freecadweb.org/viewtopic. ... 20#p177094

it is a very long thread with lots of options discussed. I was probably thinking the possibility of getting an appimage from the travis ci builds that are nevertheless done on each PR. Let's bring somebody from there in:

... i like appimages...

... i like compiling ...

Do you think it is feasible to link that previous work (or other work), so that we can produce appimages:

a) On demand from our FreeCAD forks
b) Automatically if we PR the FreeCAD official repo

[sgrogan]

Let's bring somebody from there in:

See also here: https://forum.freecadweb.org/viewtopic. ... 06#p206768
and here: https://github.com/FreeCAD/FreeCAD-AppI ... travis.yml

This is of of course a great idea. Right now we use the FreeCAD-AppImage repo to build an AppImage from the Ubuntu PPA(14.04) .deb package every 24 hours.
In the first link @realthunder has made a script to build his custom branch with pbuilder locally and pull the dependencies from the PPA. I've only quickly skimmed the script but I think it could be generalized to build any custom branch.

Automating this with Travis has some issues.
Deploying every pull request as an AppImage would probably be overkill. Maybe we can do something with git tags to tag a PR for deployment? We would have to build a .deb binary as part of the Travis process, we don't do this now.

Building FreeCAD takes longer than the allotted time on Travis if the cache needs to be refreshed, this is most likely to happen with the stuff we'd most like to test. The FreeCAD project has been graciously granted extended time by the Travis people, but most developers don't have this extension.

I think a good first step would be to test/customize/generalize @realthunders script, then try to automate after that.

On a related topic I've been intermittently working on a FreeCAD-testing PPA. The idea is the same, but my focus here is the Py3 port.

[Kunda1]

the real appimage expert

[triplus]

Hi Abdullah.

Over the years being involved with the PPA effort i gained some experience in this area. And based on that experience highly likely i wouldn't push for AppImage in the first place if it wouldn't work the way it works now. That is being fully automated and low maintenance solution build on top of already ongoing efforts. Like PPA, Travis, work done by AppImage devs ... That strategy turned out to be successful and it meet the expectations. And most importantly it got done.

If i do some math there are around 10 people (occasionally more people step up and offer help working on some specific task) involved in providing seamless delivery of FreeCAD binaries cross platforms. And at least 4 service providers we use directly to achieve that. All in all we i guess are already pushing the limits a bit. And i for example will be happy if things stay on such levels in the future and won't degrade.

But sure as everything this area can evolve in the future. And i guess i am optimistic that will happen. Likely we won't have binaries available for all pull requests tomorrow. But as creating AppImages for some dev branches is trending ATM that i guess is a nice start!

[abdullah]

Hi all,

I think I was too enthusiastic with this topic.

Somehow I expected that the route would be PR and travis, because travis already does all the building, including for PRs and it is a pity not to reuse this work further. I know about the time limitations of travis ci, as I tried to make it work with my github repo and well, time was always up. I am grateful to travis people for giving extra time for FreeCAD. But as said, we are using a lot of their resources and it sounds like we could get more out of it.

I am somehow surprised that the PPA is the actual source for the appimage. I have run my own ppa of my repo for a short time in the past. However, it is not practical for PR testing. It requires updating the branch name everytime you want a new branch or maintaining a "parallel master to FC" which is... well, I do not like to duplicate work. I guess this route is more for a branch that maintains over time in parallel with FC, having a longer development cycle than features. Generally a relatively complex new feature is done in around 1 month (e.g. sketcher support for parabola, ellipse, hyperbola, bspline). Simpler features generally span 1-2 weeks. Setting a parallel development, deployment system looks like overkilling.

Somehow my initial idea was more like reusing work that is done anyway in my wishful thinking that it could be just adding a small delta to what we already have today.

My felt need is: Getting more feedback during development. I believe there is an opportunity for improvement there. I guess time may solve this one.

Thanks to all who replied. I appreciate it.

[sgrogan]

it is a pity not to reuse this work further

Well the AppImage is built from a debian binary, which is why we re-use the binary created by the PPA. In theory we could switch the Travis build to use pbuilder and build a debian binary package there. I don't know if we could use the cache in a pbuilder environment though.

[abdullah]

I think a good first step would be to test/customize/generalize @realthunders script, then try to automate after that.

This is actually a rather good intermediate option, as it may be reasonably adapted for any branch.

abdullah wrote: ↑
Wed Jan 03, 2018 7:51 pm
it is a pity not to reuse this work further

Well the AppImage is built from a debian binary, which is why we re-use the binary created by the PPA. In theory we could switch the Travis build to use pbuilder and build a debian binary package there. I don't know if we could use the cache in a pbuilder environment though.

I know very little about this, but I guess we are using the deb from PPA to use an old ubuntu (trusty?) so as to have better support. I am unsure what "version" is actually used in travis. I also guess that it is cheaper to use the PPA because we are going to package freecad-daily anyway (even if we make a deb package with travis, assuming it is possible time/cache-wise). Of course, the whole matter changes when talking about deploying PRs (or certain PRs). Then travis has already made the work. There is a lot I would have to learn for undertaking this...

[NormandC]

I have run my own ppa of my repo for a short time in the past. However, it is not practical for PR testing. It requires updating the branch name everytime you want a new branch or maintaining a "parallel master to FC" which is...

Not sure if sgrogan agrees, but it seems to me that PPA management has been simpler since the switch from Bazaar to Git.