Ubuntu PPA Maintenance Coordination

Having trouble installing or compiling FreeCAD? Get help here.
User avatar
sgrogan
Posts: 5530
Joined: Wed Oct 22, 2014 5:02 pm

Re: Ubuntu PPA Maintenance Coordination

Postby sgrogan » Wed Apr 27, 2016 10:03 pm

NormandC wrote:Sorry if I'm not making much progress with the gitpackaging branch, I'm still trying to come to grips with Git.

I'm calling it a night...
Time is a precious thing ;)
Git is definitely different than bazaar. When using bzr I find myself translating in my head, I'm sure your experiencing the opposite.

If you want, I can incorporate the changes and push a branch.
I know:
1) doing something for boost for precise in the control file
2) copying the versioning in the rules file
3) Updating the changelog (to point back to the 0.16 release)
I don't know:
1) What, if anything, needs to be done for xenial. I've been trying to follow, but it's still over my head.

Good news is that I built the releases/FreeCAD-0-16 branch for trusy on my personal PPA (without versioning) and Launchpad didn't choke on the /
https://launchpad.net/~clhclh/+archive/ ... /+packages (recipe slashtest)
User avatar
NormandC
Posts: 18534
Joined: Sat Feb 06, 2010 9:52 pm
Location: Québec, Canada

Re: Ubuntu PPA Maintenance Coordination

Postby NormandC » Thu Apr 28, 2016 2:47 am

sgrogan wrote:Time is a precious thing ;)
Tell me about it. :?
sgrogan wrote:When using bzr I find myself translating in my head, I'm sure your experiencing the opposite.
You can say that! To me bzr is simpler and straightforward.
sgrogan wrote:If you want, I can incorporate the changes and push a branch.
No need, I managed to do it on my own. Last night, I just didn't know for sure how to push branches to Launchpad. I will need to learn how to create remotes, because pushing to LP with long git urls is becoming tedious...

Everything is here:
https://code.launchpad.net/~freecad-mai ... tpackaging

Here's the rationale behind all those branches:
  • dailybuild is the branch containing specific files modified to create freecad-daily, freecad-daily-doc and freecad-daily-dev packages. This aims to comply with item #13 in the todo list. Not functional: see below
  • master should contain the latest and up-to-date debian folder and files to build freecad debian packages from source or releases branch. The idea is to "branch it out" to the release branch when a new FreeCAD release is ready; also, the aim is to make it so we can eventually make a pull request upstream to update the /package/debian folder on Github. So, no "freecad-daily" packages here.
  • release is the branch for building from the releases/FreeCAD-0-16 branch. It should be compatible with Ubuntu 14.04, 15.10 and 16.04. When a new releases/FreeCAD-0-17 branch is created, then we simply update the files in the release branch. I believe this could be done with a git rebase from master?
  • release-precise has the same purpose as above, but only for Ubuntu 12.04. To use the release branch, I would have needed to edit the debian/control file to support libboost libraries lower than 1.54. I'm sure it could have been done, but I was too lazy to look into it. Also it will be used only for 0.16 since it's been decided to drop precise for 0.17.
I just attempted to build a freecad-daily debian package from master along with gitpackaging/dailybuild.

It almost completed! :) But it failed at the very end with this error: :(

Code: Select all

   dh_install -O--buildsystem=cmake -O--parallel
   dh_installdocs -O--buildsystem=cmake -O--parallel
   debian/rules override_dh_installchangelogs
make[1]: Entering directory `/tmp/buildd/freecad-daily-0.17~alpha1'
dh_installchangelogs ChangeLog.txt
make[1]: Leaving directory `/tmp/buildd/freecad-daily-0.17~alpha1'
   dh_installman -O--buildsystem=cmake -O--parallel
   dh_python2 -O--buildsystem=cmake -O--parallel
   dh_installmenu -O--buildsystem=cmake -O--parallel
   debian/rules override_dh_installmime
make[1]: Entering directory `/tmp/buildd/freecad-daily-0.17~alpha1'
dh_installmime
dh_install debian/mime/freecad-thumbnailer usr/bin
dh_install debian/mime/freecad.thumbnailer usr/share/thumbnailers
dh_install debian/mime/freecad.schemas etc/gconf/schemas
make[1]: Leaving directory `/tmp/buildd/freecad-daily-0.17~alpha1'
   dh_lintian -O--buildsystem=cmake -O--parallel
   dh_gconf -O--buildsystem=cmake -O--parallel
   debian/rules override_dh_icons
make[1]: Entering directory `/tmp/buildd/freecad-daily-0.17~alpha1'
install -m 644 debian/freecad/usr/share/freecad/freecad-icon-16.png debian/freecad/usr/share/icons/hicolor/16x16/apps/freecad.png
install: cannot stat 'debian/freecad/usr/share/freecad/freecad-icon-16.png': No such file or directory
make[1]: *** [override_dh_icons] Error 1
make[1]: Leaving directory `/tmp/buildd/freecad-daily-0.17~alpha1'
make: *** [binary] Error 2
dpkg-buildpackage: error: fakeroot debian/rules binary gave error exit status 2
E: Failed autobuilding of package
I: unmounting dev/pts filesystem
I: unmounting run/shm filesystem
I: unmounting proc filesystem
I: cleaning the build env 
I: removing directory /var/cache/pbuilder/build//25975 and its subdirectories
I neglected to rename the paths for application icons. Will have to look into it another day.
sgrogan wrote:1) doing something for boost for precise in the control file
What would that be? The 0.16.6700 packages were built with the default libboost*-dev packages and it seemed to work. See the control file in the release-precise branch.
sgrogan wrote:I don't know:
1) What, if anything, needs to be done for xenial. I've been trying to follow, but it's still over my head.
Mateusz provided a script to add in the debian/rules file here: viewtopic.php?f=4&t=15222&start=50#p122808

What it does is first to detect if the target system for the package is xenial; for that it uses a terminal command from the lsb-release package (added to debian/control Build-Depends). I mentioned this command in this post, try it for yourself: viewtopic.php?f=4&t=15222&start=50#p122803

Then it determines the system architecture (amd64 or i386), searches for the invalid /usr/lib/libfreeimage.so link during the build and replaces it with the proper link. So it works around the bug in the liboce-visualization-dev xenial package, without actually needing to rebuild the package.

I've added it to all branches in gitpackaging except for release-precise. It needs to be tested.
Last edited by NormandC on Thu Apr 28, 2016 4:05 am, edited 4 times in total.
User avatar
NormandC
Posts: 18534
Joined: Sat Feb 06, 2010 9:52 pm
Location: Québec, Canada

Re: Ubuntu PPA Maintenance Coordination

Postby NormandC » Thu Apr 28, 2016 3:23 am

sgrogan wrote:Good news is that I built the releases/FreeCAD-0-16 branch for trusy on my personal PPA (without versioning) and Launchpad didn't choke on the /
https://launchpad.net/~clhclh/+archive/ ... /+packages (recipe slashtest)
Great, but the version number is wrong (0.17~alpha). :D

To build 0.16 packages from a git-recipe, the recipe will have to extract the version number directly from the debian/changelog. It would be nice if it extracted it from the src/Build/Version.h file in gitversioning/releases/FreeCAD-0-16, but that's another thing I don't know how to do, and I'm not even sure it's possible.

See Version numbers and substitution variables (Launchpad Help)

So the recipe for 0.16 would read as follows:

Code: Select all

# git-build-recipe format 0.4 deb-version 1:{debversion:gitpackaging}
lp:~freecad-maintainers/+git/gitmaster releases/FreeCAD-0-16
merge gitpackaging lp:~freecad-maintainers/+git/gitpackaging release
merge gitversioning lp:~freecad-maintainers/+git/gitversioning releases/FreeCAD-0-16
The debversion variable extracts the full version number between parentheses:

Code: Select all

freecad (1:0.16.6703-1ppa1) xenial; urgency=low
Launchpad will automatically append ~ubuntu14.04.1 for the trusty package, ~ubuntu16.04.1 for the xenial package, etc. That's what it does with the daily builds packages.
User avatar
NormandC
Posts: 18534
Joined: Sat Feb 06, 2010 9:52 pm
Location: Québec, Canada

Re: Ubuntu PPA Maintenance Coordination

Postby NormandC » Thu Apr 28, 2016 3:37 am

I created the gitmaster-release recipe based on my previous reply.

I requested a build, but the recipe failed to build. https://launchpadlibrarian.net/25682832 ... log.txt.gz

Code: Select all

Building recipe:
# git-build-recipe format 0.4 deb-version 1:{debversion:gitpackaging}
lp:~freecad-maintainers/+git/gitmaster releases/FreeCAD-0-16
merge gitpackaging lp:~freecad-maintainers/+git/gitpackaging release
merge gitversioning lp:~freecad-maintainers/+git/gitversioning releases/FreeCAD-0-16

RUN ['git-build-recipe', '--safe', '--no-build', '--manifest', '/home/buildd/build-RECIPEBRANCHBUILD-1126690/chroot-autobuild/home/buildd/work/tree/manifest', '--distribution', 'trusty', '--allow-fallback-to-native', '--append-version', '~ubuntu14.04.1', '/home/buildd/build-RECIPEBRANCHBUILD-1126690/chroot-autobuild/home/buildd/work/recipe', '/home/buildd/build-RECIPEBRANCHBUILD-1126690/chroot-autobuild/home/buildd/work/tree']
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/gitbuildrecipe/recipe.py", line 348, in merge_branch
    child_branch.commit)
  File "/usr/lib/python3/dist-packages/gitbuildrecipe/recipe.py", line 637, in git_call
    **kwargs)
  File "/usr/lib/python3.4/subprocess.py", line 620, in check_output
    raise CalledProcessError(retcode, process.args, output=output)
subprocess.CalledProcessError: Command '['git', '-C', '/home/buildd/build-RECIPEBRANCHBUILD-1126690/chroot-autobuild/home/buildd/work/tree/recipe', 'merge', '--commit', '-m', 'Merge release', '58b46940c102d19fc61aa9b991583c7bcc1b0129']' returned non-zero exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/git-build-recipe", line 9, in <module>
    load_entry_point('git-build-recipe==0.1', 'console_scripts', 'git-build-recipe')()
  File "/usr/lib/python3/dist-packages/gitbuildrecipe/main.py", line 110, in main
    build_tree(base_branch, working_directory)
  File "/usr/lib/python3/dist-packages/gitbuildrecipe/recipe.py", line 464, in build_tree
    _build_inner_tree(base_branch, target_path)
  File "/usr/lib/python3/dist-packages/gitbuildrecipe/recipe.py", line 388, in _build_inner_tree
    instruction.apply(target_path)
  File "/usr/lib/python3/dist-packages/gitbuildrecipe/recipe.py", line 520, in apply
    merge_branch(self.recipe_branch, target_path)
  File "/usr/lib/python3/dist-packages/gitbuildrecipe/recipe.py", line 350, in merge_branch
    raise MergeFailed(e.output)
gitbuildrecipe.recipe.MergeFailed: git merge failed:
Auto-merging .gitignore
CONFLICT (add/add): Merge conflict in .gitignore
Automatic merge failed; fix conflicts and then commit the result.

resource.struct_rusage(ru_utime=43.583902, ru_stime=9.1439, ru_maxrss=174384, ru_ixrss=0, ru_idrss=0, ru_isrss=0, ru_minflt=257698, ru_majflt=41, ru_nswap=0, ru_inblock=10872, ru_oublock=1159168, ru_msgsnd=0, ru_msgrcv=0, ru_nsignals=0, ru_nvcsw=77937, ru_nivcsw=30042)
RUN: /usr/share/launchpad-buildd/slavebin/scan-for-processes ['scan-for-processes', 'RECIPEBRANCHBUILD-1126690']
Scanning for processes to kill in build /home/buildd/build-RECIPEBRANCHBUILD-1126690/chroot-autobuild...
RUN: /usr/share/launchpad-buildd/slavebin/umount-chroot ['umount-chroot', 'RECIPEBRANCHBUILD-1126690']
Unmounting chroot for build RECIPEBRANCHBUILD-1126690...
RUN: /usr/share/launchpad-buildd/slavebin/remove-build ['remove-build', 'RECIPEBRANCHBUILD-1126690']
Removing build RECIPEBRANCHBUILD-1126690
What now. Why the heck does it dislike my .gitignore file. :?
User avatar
NormandC
Posts: 18534
Joined: Sat Feb 06, 2010 9:52 pm
Location: Québec, Canada

Re: Ubuntu PPA Maintenance Coordination

Postby NormandC » Thu Apr 28, 2016 3:54 am

Copied .gitignore file from gitmaster to resolve the merge conflict. My .gitignore file in gitpackaging didn't have the same content as the one in gitmaster.

Still fails, but for a new reason.

Code: Select all

Building recipe:
# git-build-recipe format 0.4 deb-version 1:{debversion:gitpackaging}
lp:~freecad-maintainers/+git/gitmaster releases/FreeCAD-0-16
merge gitpackaging lp:~freecad-maintainers/+git/gitpackaging release
merge gitversioning lp:~freecad-maintainers/+git/gitversioning releases/FreeCAD-0-16

RUN ['git-build-recipe', '--safe', '--no-build', '--manifest', '/home/buildd/build-RECIPEBRANCHBUILD-1126691/chroot-autobuild/home/buildd/work/tree/manifest', '--distribution', 'trusty', '--allow-fallback-to-native', '--append-version', '~ubuntu14.04.1', '/home/buildd/build-RECIPEBRANCHBUILD-1126691/chroot-autobuild/home/buildd/work/recipe', '/home/buildd/build-RECIPEBRANCHBUILD-1126691/chroot-autobuild/home/buildd/work/tree']
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/gitbuildrecipe/deb_util.py", line 155, in add_autobuild_changelog_entry
    changelog.Version(version)
  File "/usr/lib/python3/dist-packages/debian/debian_support.py", line 213, in __init__
    super(AptPkgVersion, self).__init__(version)
  File "/usr/lib/python3/dist-packages/debian/debian_support.py", line 111, in __init__
    self.full_version = version
  File "/usr/lib/python3/dist-packages/debian/debian_support.py", line 137, in __setattr__
    self._set_full_version(str(value))
  File "/usr/lib/python3/dist-packages/debian/debian_support.py", line 116, in _set_full_version
    raise ValueError("Invalid version string %r" % version)
ValueError: Invalid version string '1:{debversion:gitpackaging}~ubuntu14.04.1'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/git-build-recipe", line 9, in <module>
    load_entry_point('git-build-recipe==0.1', 'console_scripts', 'git-build-recipe')()
  File "/usr/lib/python3/dist-packages/gitbuildrecipe/main.py", line 147, in main
    append_version=args.append_version)
  File "/usr/lib/python3/dist-packages/gitbuildrecipe/deb_util.py", line 157, in add_autobuild_changelog_entry
    raise Exception("Invalid deb-version: %s: %s" % (version, e))
Exception: Invalid deb-version: 1:{debversion:gitpackaging}~ubuntu14.04.1: Invalid version string '1:{debversion:gitpackaging}~ubuntu14.04.1'
resource.struct_rusage(ru_utime=41.552658, ru_stime=10.205626, ru_maxrss=174652, ru_ixrss=0, ru_idrss=0, ru_isrss=0, ru_minflt=247029, ru_majflt=41, ru_nswap=0, ru_inblock=10984, ru_oublock=1163816, ru_msgsnd=0, ru_msgrcv=0, ru_nsignals=0, ru_nvcsw=75194, ru_nivcsw=40146)
RUN: /usr/share/launchpad-buildd/slavebin/scan-for-processes ['scan-for-processes', 'RECIPEBRANCHBUILD-1126691']
Scanning for processes to kill in build /home/buildd/build-RECIPEBRANCHBUILD-1126691/chroot-autobuild...
RUN: /usr/share/launchpad-buildd/slavebin/umount-chroot ['umount-chroot', 'RECIPEBRANCHBUILD-1126691']
Unmounting chroot for build RECIPEBRANCHBUILD-1126691...
RUN: /usr/share/launchpad-buildd/slavebin/remove-build ['remove-build', 'RECIPEBRANCHBUILD-1126691']
Removing build RECIPEBRANCHBUILD-1126691
I don't understand why it cannot extract the version from {debversion:gitpackaging}. We could always simply replace it with 1:0.16.6703 in the recipe but this would be prone to errors, I wish for the darn variable to work. :x

EDIT: sometimes you have to let go. :roll: I edited the recipe to:

Code: Select all

# git-build-recipe format 0.4 deb-version 1:0.16.6703-1ppa1
Source packages built successfully, binary packages are currently building. If they succeed, they will be copied to my PPA.
https://launchpad.net/~gemnoc/+archive/ubuntu/ppa
User avatar
f3nix
Posts: 313
Joined: Sat May 30, 2015 11:58 am

Re: Ubuntu PPA Maintenance Coordination

Postby f3nix » Thu Apr 28, 2016 4:32 am

Hi Normand!

This seems to work for me:

Code: Select all

# git-build-recipe format 0.4 deb-version {debversion}
lp:~freecad-maintainers/+git/gitmaster releases/FreeCAD-0-16
merge gitpackaging lp:~freecad-maintainers/+git/gitpackaging release
merge gitversioning lp:~freecad-maintainers/+git/gitversioning releases/FreeCAD-0-16
https://code.launchpad.net/~f3nix/+reci ... -daily-ver

HTH. :)

Cheers,
Mateusz

EDIT: The last build error is from testing this:

Code: Select all

# git-build-recipe format 0.4 deb-version {debversion:gitpackaging}
User avatar
NormandC
Posts: 18534
Joined: Sat Feb 06, 2010 9:52 pm
Location: Québec, Canada

Re: Ubuntu PPA Maintenance Coordination

Postby NormandC » Thu Apr 28, 2016 5:06 am

Hi Mateusz,

I don't understand why it works, when not specifying the correct branch? :?
All variables other than time are derived from a particular branch. By default they use the base branch (eg. {revno}), but they can also use a named branch (eg. {revno:packaging}).

debversion, debupstream and debupstream-base require debian/changelog to exist in the given branch. For recipe versions 0.4 and newer, you must specify the name of the branch (e.g. {debupstream-base:packaging}).
Maybe it's because we're using merge here rather than nest-part, so the packaging branch "disappears"?

Thanks for finding the fix! :)

BTW my packages successfully built, and they are functional (on amd64 anyway) as well as show the correct version in About FreeCAD. But they should not be copied to the Stable PPA: I forgot again to set a PPA dependency to the Stable or Daily PPA, so FEM is non functional (no netgen-headers nor calculix), and are built with the prehistoric liboce from the trusty repo.

@ sgrogan: if you have time tomorrow (today :? ) and are willing to take it from here for the stable packages, you're welcome to do so. Otherwise I'll resume in about 20 hours. What's needed is to update the .gitignore file in all branches in gitpackaging so it is identical to the one in gitmaster.

Then the precise 0.16 package will need a separate recipe.
User avatar
sgrogan
Posts: 5530
Joined: Wed Oct 22, 2014 5:02 pm

Re: Ubuntu PPA Maintenance Coordination

Postby sgrogan » Thu Apr 28, 2016 11:54 am

NormandC wrote: @ sgrogan: if you have time tomorrow (today :? ) and are willing to take it from here for the stable packages, you're welcome to do so. Otherwise I'll resume in about 20 hours. What's needed is to update the .gitignore file in all branches in gitpackaging so it is identical to the one in gitmaster.
I will be able to pick this up in about 9 hours. I'll post my progress as I go. I very much like the branch structure you have set up :)
User avatar
sgrogan
Posts: 5530
Joined: Wed Oct 22, 2014 5:02 pm

Re: Ubuntu PPA Maintenance Coordination

Postby sgrogan » Thu Apr 28, 2016 9:25 pm

I've updated the .gitiignores on all branches.
NormandC wrote: Then the precise 0.16 package will need a separate recipe.
Should we go for this on FreeCAD Maintainers or one more round on my personal PPA?
EDIT: Tried to stage the recipes, but forgot to check build on request so they are building daily. So far so good. https://launchpad.net/~freecad-maintain ... /+packages
EDIT2: 1:0.16.6703-1ppa1~ubuntu14.04.1 < 1:0.16+bzr5099+201604072313~ubuntu14.04.1 :(
User avatar
NormandC
Posts: 18534
Joined: Sat Feb 06, 2010 9:52 pm
Location: Québec, Canada

Re: Ubuntu PPA Maintenance Coordination

Postby NormandC » Fri Apr 29, 2016 2:12 am

sgrogan wrote:EDIT: Tried to stage the recipes, but forgot to check build on request so they are building daily. So far so good.
Cool! Thanks. :) But
https://code.launchpad.net/~freecad-mai ... se-precise

(What a long recipe name! :D)

Code: Select all

# git-build-recipe format 0.4 deb-version 1:0.16.6703-1ppa1
As per my discussion with Mateusz late last night, you could have used

Code: Select all

# git-build-recipe format 0.4 deb-version 1:{debversion}
Which worked. I'm now pretty sure the reason {debversion:gitpackaging} didn't work is because the git recipe uses "merge" instead of "nest-part". As I understand it, a git merge fuses all branches into one, so there is no more "gitpackaging" branch when Launchpad attempts to figure out the debversion variable. Bzr's nest-part doesn't work the same way.
sgrogan wrote:EDIT2: 1:0.16.6703-1ppa1~ubuntu14.04.1 < 1:0.16+bzr5099+201604072313~ubuntu14.04.1 :(
I don't have this problem here. Both Stable and Daily PPAs are in my software sources. If I delete my freecad package, my package manager (Synaptic) proposes 1:0.16.6703 version.

If using the terminal (sudo apt-get install freecad), the 1:0.16.6703 package is installed automatically.

Those version conflicts are frustrating, aren't they? But at this point, we'll have to trust users to deal with it on their own, or ask for help on the forum. It's a simple thing to uninstall one and install the other. We cannot always hold the end user's hand. After all Windows users need to uninstall FreeCAD 0.15, the 0.16 installer does not do it for them.

And I do not want to bump the epoch number yet again. Let's not abuse of this thing.

In any case, when we switch the Daily PPA to a "freecad-daily" package, users will have no choice but to install it manually. We'll make an annoucement on the forum, but I'm sure some users will always come back weeks or months later and wonder why the Daily Builds has been dead for months...
cron