Ubuntu PPA Maintenance Coordination

Having trouble installing or compiling FreeCAD? Get help here.
User avatar
NormandC
Posts: 12031
Joined: Sat Feb 06, 2010 9:52 pm
Location: Québec, Canada

Ubuntu PPA Maintenance Coordination

Postby NormandC » Sun Apr 10, 2016 11:19 pm

As discussed privately with sgrogan and mentioned in the 0.16 release topic, I've offered my help with the Ubuntu PPA maintenance and he accepted. While it's beneficial to get insights and links over a forum, it's a different thing altogether to do the real work (as can be seen below).

I feel that having at least two active maintainers is a goal to aim for, for redundancy but also to avoid fatigue, frustration etc.

With that said, this topic's purpose will be to coordinate actions on the multiple PPAs managed by the FreeCAD maintainers team. Work is prioritized toward the "official" 0.16 release. Done, work is now focused on reactivating the Daily Builds PPA.

(Updated Sun, 01 May 2016 15:47:19 -0400)
TODO List


  1. Update the Vcs links in the various branch's debian/control files. Note: see item 10. (done)
     
  2. Update the dependencies in the control files. libsoqt4-dev isn't required but it does pull in some dependencies that are. (DONE)
     
  3. Update oce to 0.17.1 for the 0.16 release by backporting the xenial package. (DONE)
    1. Upload first on the OCE Releases PPA. Done for precise, trusty and wily. vivid not done because it reached EOL in February.
    2. Copy the binaries to the FreeCAD Stable Releases PPA. Done for precise, trusty and wily. vivid excluded for the same reason as above.
    3. Copy the binaries to the FreeCAD Daily Builds PPA. precise, trusty & wily packages copied - DONE
  4. Updating netgen. Completing 3 is required as netgen builds against oce. (DONE)
    1. Upload the packages to the Stable PPA for precise, trusty and wily. DONE
    2. Copy the binaries to the Daily PPA for precise, trusty and wily. DONE.
  5. Backporting calculix 2.9 from xenial. Make sure the old ccx 2.7 package gets removed from the user's system. (DONE)
    1. Upload the packages to the Stable PPA for precise, trusty and wily. DONE
    2. Copy the binaries to the Daily PPA. DONE
  6. Fix the freecad-doc package which is empty at the moment. (DONE)
     
  7. Find a workaround in xenial for liboce-visualization-dev not linking libfreeimage-dev properly. It's a Multi-Arch support issue. This prevents FreeCAD from building on the Daily PPA thus we have the same problem with the 0.16 release. (DONE thanks to f3nix/Mateusz)
     
  8. Provide 0.16 release packages on the Stable Releases PPA. This is priority #1 in the list, but it depends on items 1-7 being completed first. Item 7 completion is not required to build precise, trusty & wily packages. The issue with xenial can be postponed but not for too long, 16.04 release date is April 21st. (COMPLETED ON 2016-04-19, release update uploaded on 2016-04-28)
     
  9. If no solution is found for 7, uploading an "improper" xenial 0.16 package, probably like the official ubuntu package as a work around until an upstream fix. No longer necessary
     
  10. Streamline the Daily Builds PPA process. Currently there are many packaging branches and build recipes (some of them obsolete). It may be possible to use a single packaging branch and a single build recipe with a properly tailored debian/control file (item 2). We'll know more after completing 8. (in progress)
     
  11. Split package into smaller ones (issue #857) --> goal for 0.17? (needs to be discussed with developers and other maintainers)
     
  12. Document on the wiki what we do.
     
  13. Look into the possibility of producing a self contained snapshot (like Windows pre-release builds), or making it possible to install the 0.16 release and the daily build at the same time. The 0.17 development cycle will bring disruptive changes that may come very soon. It's possible the daily builds will become less stable/usable for at least some time, on the other hand wider user feedback will be useful. (forum discussion) (second solution in progress)
Last edited by NormandC on Sun May 01, 2016 7:54 pm, edited 5 times in total.
User avatar
NormandC
Posts: 12031
Joined: Sat Feb 06, 2010 9:52 pm
Location: Québec, Canada

Re: Ubuntu PPA Maintenance Coordination

Postby NormandC » Wed Apr 13, 2016 2:21 am

NormandC wrote:6. Fix the freecad-doc package which is empty at the moment. (sgrogan)

I think I found why the freecad-doc package is empty of files.

In debian/rules http://bazaar.launchpad.net/~freecad-ma ... /rules#L17

Code: Select all

-DCMAKE_INSTALL_DOCDIR="/usr/share/doc/freecad-doc"

There should be a space and a backslash at the end of line #17.

Code: Select all

-DCMAKE_INSTALL_DOCDIR="/usr/share/doc/freecad-doc" \

See http://freecadweb.org/tracker/view.php?id=2271#c6902

I'm working on a packaging/debian/release branch with updated debian files. Now to see if I can git push it to lp:freecad.

Edit: I pushed it as a Git branch on Launchpad. https://git.launchpad.net/freecad?h=rel ... eeCAD-0-16
User avatar
NormandC
Posts: 12031
Joined: Sat Feb 06, 2010 9:52 pm
Location: Québec, Canada

Re: Ubuntu PPA Maintenance Coordination

Postby NormandC » Fri Apr 15, 2016 4:00 am

NormandC wrote:There should be a space and a backslash at the end of line #17.

That did not solve the issue.

As mentioned in this topic, I fixed it by creating a new freecad-doc.install file under the debian folder with the following content:

Code: Select all

src/Doc/freecad.qch usr/share/doc/freecad-doc
src/Doc/freecad.qhc usr/share/doc/freecad-doc

This copies the qch and qhc files from the /src/Doc folder in the tarball to /usr/share/doc/freecad-doc.

On another matter, I've been building binary packages locally in a clean environment (commonly known as "chroot") using PBuilder. The problem was that it took a very long time to compile, since it only used one thread. The rules file included this:

Code: Select all

#!/usr/bin/make -f
%:
   dh $@ --buildsystem=cmake --parallel --with python2

But it is not sufficient. Then I looked at the rules file I used to create the 0.15.4671 package last year, and it started with this:

Code: Select all

#!/usr/bin/make -f

ifeq ($(findstring parallel=,$(DEB_BUILD_OPTIONS)),)
   export DEB_BUILD_OPTIONS+=parallel=$(shell getconf _NPROCESSORS_ONLN)
endif

%:
   dh $@ --buildsystem=cmake --parallel --with python2

I remembered that I had copied it last year from the rules file in the debian oce package. I have no idea why I didn't update the daily ppa packaging branches! This makes compilation a lot faster, on my quad-core CPU it is the equivalent of "make -j4". And I remembered that the oce 0.17.1 packages compiled a lot faster than did the freecad ones.

If you type getconf _NPROCESSORS_ONLN in a terminal, it will return the number of cores (or threads?) on your PC.
User avatar
NormandC
Posts: 12031
Joined: Sat Feb 06, 2010 9:52 pm
Location: Québec, Canada

Re: Ubuntu PPA Maintenance Coordination

Postby NormandC » Fri Apr 15, 2016 4:50 am

With PartDesignNext now merged to master, it may be necessary to bump item 13 before 10 in the list.
User avatar
sgrogan
Posts: 2969
Joined: Wed Oct 22, 2014 5:02 pm

Re: Ubuntu PPA Maintenance Coordination

Postby sgrogan » Fri Apr 15, 2016 10:25 pm

NormandC wrote:

Fantastic.
NormandC wrote:With PartDesignNext now merged to master, it may be necessary to bump item 13 before 10 in the list.

I agree. Can we re-name the package? FreeCAD-daily? There will be issues with the "UserAppDAta" folder? Thinking out load but we need to make sure the two versions share a common user.cfg and system.cfg, or not?
User avatar
NormandC
Posts: 12031
Joined: Sat Feb 06, 2010 9:52 pm
Location: Québec, Canada

Re: Ubuntu PPA Maintenance Coordination

Postby NormandC » Sat Apr 16, 2016 5:32 am

sgrogan wrote:Can we re-name the package? FreeCAD-daily?

The package can be renamed. freecad-daily is as good a name as any, I was thinking freecad0.17 myself, adding "alpha" to the version number (FYI debian package names cannot contain uppercase letters see here for naming and versioning conventions).

I think it can be done without too much trouble, but I've been wrong before :D
  • Change all references to freecad in paths in debian/rules
  • changelog (using my example):

    Code: Select all

    freecad0.17 (0.17~alpha-1trusty1) trusty; urgency=low

      * daily build of freecad 0.17 Alpha.
  • control: keep "freecad" as Source name, but change package names
  • Rename files named freecad*, check paths etc.
  • Haven't found where it's set, but we need to rename the symbolic links for the freecad, freecadcmd and freecad-thumbnailer executable files that get copied to usr/bin
sgrogan wrote:There will be issues with the "UserAppDAta" folder? Thinking out load but we need to make sure the two versions share a common user.cfg and system.cfg, or not?

I'm not sure on that. Currently I have the 0.16.6700 stable package installed, and I have FreeCAD compiled from master. Both use the same ~/.FreeCAD folder with cfg files. It does not seem to create problems for now - it's quite possible that it could cause trouble further along in the 0.17 development, if the devs add more preferences settings or change the existing ones. We should also consider that users may wish to have different settings for the release package and the daily package.

As for creating a separate ~/.FreeCAD-0.17 folder, it's probably possible, I did it for a local compilation of FreeCAD 0.14, it was controlled by a custom-made launcher that had the proper command - but I deleted it last week. :oops: I think I found the info in Start up and Configuration.
Last edited by NormandC on Sat Apr 16, 2016 6:00 am, edited 3 times in total.
User avatar
NormandC
Posts: 12031
Joined: Sat Feb 06, 2010 9:52 pm
Location: Québec, Canada

Re: Ubuntu PPA Maintenance Coordination

Postby NormandC » Sat Apr 16, 2016 5:56 am

Universal debian/control file

I mentioned that I thought we could make a "universal" control file that would work on all Ubuntu versions.

I was short on time with the Stable PPA so I left it aside but here's what I have so far, it's an update on the "packaging/debian/release" branch I uploaded to LP. It needs testing. (I removed our email addresses)

Code: Select all

Source: freecad
Section: science
Priority: extra
Maintainer: Normand C <email redacted>
Uploaders: sgrogan <email redacted>
Vcs-Browser: https://git.launchpad.net/freecad
Vcs-Git: git://git.launchpad.net/freecad
Homepage: http://www.freecadweb.org/
Build-Depends:
 cmake,
 debhelper (>= 9),
 libboost-dev (>=1.55) | libboost1.55-dev,
 libboost-filesystem-dev (>=1.55) | libboost-filesystem1.55-dev,
 libboost-program-options-dev (>=1.55) | libboost-program-options1.55-dev,
 libboost-python-dev (>=1.55) | libboost-python1.55-dev,
 libboost-regex-dev (>=1.55) | libboost-regex1.55-dev,
 libboost-signals-dev (>=1.55) | libboost-signals1.55-dev,
 libboost-thread-dev (>=1.55) | libboost-thread1.55-dev,
 libcoin80-dev,
 libeigen3-dev,
 libfreeimage-dev,
 libfreetype6-dev,
 liboce-ocaf-dev,
 liboce-visualization-dev,
 libpyside-dev,
 libqt4-dev,
 libqt4-opengl-dev,
 libqtwebkit-dev,
 libshiboken-dev,
 libspnav-dev,
 libxerces-c-dev,
 libzipios++-dev,
 netgen-headers,
 oce-draw,
 pyside-tools,
 python-dev,
 python-ply,
 swig
Standards-Version: 3.9.5

Package: freecad
Architecture: any
Depends:
 python-pivy,
 python-pyside.qtcore,
 python-pyside.qtsvg,
 python-pyside.qtuitools,
 ${shlibs:Depends},
 ${misc:Depends},
 ${python:Depends}
Recommends:
 calculix-ccx,
 graphviz
Suggests:
 freecad-doc,
 python-collada,
 python-matplotlib
Description: Extensible Open Source CAx program (beta)
 FreeCAD is a general purpose feature-based, parametric 3D modeler for CAD, MCAD,
 CAx, CAE and PLM, aimed directly at mechanical engineering and product design but
 also fits a wider range of uses in engineering, such as architecture or other engineering
 specialties. It is 100% Open Source (LGPL2+ license) and extremely modular, allowing
 for very advanced extension and customization.
 .
 FreeCAD is based on OpenCasCade, a powerful geometry kernel, features an Open Inventor-compliant
 3D scene representation model provided by the Coin 3D library, and a broad Python API.
 The interface is built with Qt. FreeCAD runs exactly the same way on Windows, Mac OSX, BSD and
 Linux platforms.

Package: freecad-dev
Architecture: any
Section: libdevel
Depends:
 ${shlibs:Depends},
 ${misc:Depends},
 freecad (= ${binary:Version})
Description: FreeCAD development files
 FreeCAD is a general purpose feature-based, parametric 3D modeler for CAD, MCAD,
 CAx, CAE and PLM, aimed directly at mechanical engineering and product design but
 also fits a wider range of uses in engineering, such as architecture or other engineering
 specialties. It is 100% Open Source (LGPL2+ license) and extremely modular, allowing
 for very advanced extension and customization.
 For more details see http://www.freecadweb.org/
 .
 This package contains headers and symlinks necessary to
 develop modules for FreeCAD.

Package: freecad-doc
Architecture: all
Section: doc
Depends:
 ${misc:Depends},
 qt4-dev-tools
Description: FreeCAD documentation
 FreeCAD is a general purpose feature-based, parametric 3D modeler for CAD, MCAD,
 CAx, CAE and PLM, aimed directly at mechanical engineering and product design but
 also fits a wider range of uses in engineering, such as architecture or other engineering
 specialties. It is 100% Open Source (LGPL2+ license) and extremely modular, allowing
 for very advanced extension and customization.
 For more details see http://www.freecadweb.org/
 .
 This package contains the FreeCAD documentation.
 .
 The documentation is provided in Qt's new help format;
 the new help format version can be viewed in conjunction with the Qt Assistant
 found in the qt4-dev-tools package.

The Vcs-Browser and Vcs-Git urls are if we manage to properly set up lp:freecad. I haven't been able to so far, so we may need to use a freecad-maintainers url instead.

In the "Maintainer" field, the best would have been to put the FreeCAD Maintainers team and put me in the Uploaders, but we don't have an email address for it and debian policy requires an email (I'm sure lack of it would prevent the build).

As for the libboost libraries: (>=1.55) ensures no lower version is used, which automatically disqualifies building on precise. The use of the pipe character | allows for an alternate package, here the trusty ones. precise was the only Ubuntu version using libcoin60 so I didn't add it as an alternate package.
User avatar
sgrogan
Posts: 2969
Joined: Wed Oct 22, 2014 5:02 pm

Re: Ubuntu PPA Maintenance Coordination

Postby sgrogan » Sat Apr 16, 2016 5:09 pm

Thanks for the info NormandC
NormandC wrote:The Vcs-Browser and Vcs-Git urls are if we manage to properly set up lp:freecad. I haven't been able to so far, so we may need to use a freecad-maintainers url instead.

I need to research this more. At the moment exactly what these links do is over my head. Modifying them is when Launchpad began not being able to find the changelog. Although I modified the changelog at the same time.
User avatar
NormandC
Posts: 12031
Joined: Sat Feb 06, 2010 9:52 pm
Location: Québec, Canada

Re: Ubuntu PPA Maintenance Coordination

Postby NormandC » Sat Apr 16, 2016 5:45 pm

sgrogan wrote: At the moment exactly what these links do is over my head. Modifying them is when Launchpad began not being able to find the changelog. Although I modified the changelog at the same time.

I've given it some thought.

When in doubt, always refer to the Debian Maintainer's guide. :D
https://www.debian.org/doc/manuals/main ... ex.en.html

When it doesn't answer your question, you have to bite the bullet and refer to the Debian Policy Manual. :?
https://www.debian.org/doc/debian-policy/

As for the Vcs-Browser and Vcs-Git fields, see https://www.debian.org/doc/debian-polic ... VCS-fields
Debian source packages are increasingly developed using VCSs. The purpose of the following fields is to indicate a publicly accessible repository where the Debian source package is developed.

I think this means I/we have not been using the Vcs fields the way they're supposed to. We've put the FreeCAD project's Git repo, but that's not where the debian source package is developed. It may be best to revert to the original fields from the Debian and Ubuntu control file, which can be accessed here: http://anonscm.debian.org/cgit/debian-s ... eecad.git/

Unless as I said we manage to set up a Git repo on Launchpad. I think we should try to replicate what the Debian Science Maintainers do with their Git repo. I'll post a different reply to discuss this further.

In the control file, you put an http link in Vcs-Git, but it expects a git url (git://). Debian policy is really, really fussy, maybe that's the reason for the error.

On the other hand it may not be related. Who knows. :D
User avatar
NormandC
Posts: 12031
Joined: Sat Feb 06, 2010 9:52 pm
Location: Québec, Canada

Re: Ubuntu PPA Maintenance Coordination

Postby NormandC » Sat Apr 16, 2016 7:21 pm

Ubuntu PPA source package development

(Sorry if I'm repeating what we've already discussed this in the [For Ubuntu PPA Maintainers] Uploading a Stable Release topic, but I think this is a more centralized place to discuss the whole PPA scope.)

On Launchpad, what we deal with is source packages. Source packages are recipes to build binary packages, which are the *.deb packages that can be installed from the console or in a package manager.

The imported bzr branch named "master" used to build the Daily Builds PPA doesn't sync anymore due to the Git signed commit bug. It is highly unlikely this will ever be fixed by the Launchpad people (the report is 3 ½ years old). Unless I'm missing something obvious, we'll have to push updates to Lanchpad manually, just like we do on the packaging branches.

With Launchpad now supporting Git repositories (still in Beta though), I believe it may be desirable to switch to a Git-only work flow rather than to continue using bzr branches.

I see a lot of advantages in this, the main one being, the FreeCAD developers and advanced users here know Git. None of them knows bazaar. With Git we can get support if we get into trouble. The other potential advantage would be that with a package built directly from a cloned Git repo rather than a bzr repo, we may no longer require a Versioning.git branch.

As for automating builds: it would be possible on Linux to write a script that pulls from freecad's github master, then pushes it to the Launchpad Git. This script could then be automated with cron, assuming one of us decides to keep a PC on permanently. Otherwise we just have to launch the script once a day which is no big deal.

The Debian Science Maintainers use Git, and I'm sure they know their stuff. This is where they develop source packages for FreeCAD: http://anonscm.debian.org/cgit/debian-s ... eecad.git/

In the summary view, you can see the following branches:
  • experimental: the repository where they test packages. Advanced debian users can install packages from the experimental repository.
  • jessie-backports: the backports repository for Debian Jessie users.
  • master: where their own debian development happens.
  • upstream: contains only the original FreeCAD source code of a specific release branch minus the "package" folder. It looks like they import a DFSG tarball (there's a script to do that in FreeCAD's source code).
All mentioned branches except upstream contain the freecad source code with a debian folder at its root (not in /package/debian).

If we try to replicate this work flow on Launchpad, and based on my failed experiments of the past week :D this is what I think would need to be set up:
  • The master branch needs to be like the Debian Science Maintainers one, with a "debian" folder at the root. Let's not make the mistake I did and get a "detached HEAD" because of a non-existent master!
  • The freecad master has to be a secondary branch, not the Launchpad Git repo master, and needs to be renamed somehow. "upstream" is an obvious choice, any other ideas welcome.
  • Each time we git pull freecad master into upstream, we then merge into the Launchpad master. Whether this would mess up the commits therefore the Version.h file, I don't know.
  • releases/FreeCAD-0-xx branches could be imported as is.
  • separate branches for freecad-release and freecad-daily, if we're going to have a different naming scheme for the daily builds.
  • The "freecad-release" branch would be a merge with the latest releases/FreeCAD-0-xx branch and be used to upload to the Stable Releases PPA.
  • The "freecad-daily" branch would be a merge with Launchpad master to build Daily Builds PPA packages.
  • We would no longer need to create and manage separate packaging branches, as everything would happen inside the Git repo.
So what do you think sgrogan? Does it make sense?

Advice from the Git experts would also be welcome.

Links