GSoC '18: CM Project Thread

[kkremitzki]

OCCT 7.3 is now in Debian NEW, since this is just an upgrade it shouldn't be long before it's available in experimental.

Good news: I managed to get 2-and-3 building working for pivy!
Bad news: I found out that what I thought was a 2-and-3 build for gmsh actually had python 3 .so's pointed at a libgmsh3 built against python 2, so it's actually either-or, like netgen is currently.

On the bright side, by the time I can fix the gmsh 2-and-3 problem, I should be learnèd enough to do the same thing for netgen and freecad.

[NormandC]

OCCT 7.3 is now in Debian NEW

Sweet! I gather that you haven't made the patch to enable exception handling yet?

https://salsa.debian.org/science-team/o ... an/patches

[kkremitzki]

OCCT 7.3 is now in Debian NEW

Sweet! I gather that you haven't made the patch to enable exception handling yet?

https://salsa.debian.org/science-team/o ... an/patches

I just put it up at https://salsa.debian.org/science-team/o ... requests/1.

Since Tobias just uploaded the first Debian revision 7.3.0+dfsg1-1, we have to wait for the NEW queue and buildds to have a go at the package; since we are tracking the shared library symbols across all the Debian architectures, there may be some failures on various architectures. Depending upon how that goes, the change will either make it in via another attempt at the -1 revision, or if they all succeed, it'll be in -2.

tl;dr: the package is in a very temporary limbo.

[NormandC]

Great. I might have some time in the coming week to look into stealing your OCCT work (again) for the benefit of the FreeCAD PPA.

[yorik]

Great job Kurt!

[kkremitzki]

Great. I might have some time in the coming week to look into stealing your OCCT work (again) for the benefit of the FreeCAD PPA.

Don't rush! I am going to be working on it today and Friday.

---
Repo dump!

https://salsa.debian.org/kkremitzki-guest/freecad
https://salsa.debian.org/kkremitzki-guest/netgen
https://salsa.debian.org/kkremitzki-guest/gmsh
https://salsa.debian.org/kkremitzki-guest/pivy

There ended up being a bit more to the cleanup than I was expecting. Besides the always-there "just one more thing...", I also ran into two interesting problems:

- tar, of all packages, was breaking the importing of new code in the git-buildpackage workflow, and even though dpkg -l says my version isn't the one affected, my Ubuntu 18.04 VMs (compared to my Debian Sid ones) seemed to be having it

- the latest version of libmed, 3.1.3, was actually packaged into experimental 2 weeks ago, and none of our dependencies seem to build against it correctly at the moment. Since I had been using semi-dirty mixed unstable & experimental build environments, I only started seeing bad behavior from this in my final cleanup stages with totally clean build environments.

- it turns out our pivy fork actually does depend on pyside2/shiboken2, and there's an additional test failure in examples/extend/scale_test.py:25, failing on from shapescale import * which I couldn't readily address. So, perhaps no Python 3 build quite yet, although now that PySide 2 has been officially released upstream, we can hopefully see it soon in Debian.

[looo]

- it turns out our pivy fork actually does depend on pyside2/shiboken2, and there's an additional test failure in examples/extend/scale_test.py:25, failing on from shapescale import * which I couldn't readily address. So, perhaps no Python 3 build quite yet, although now that PySide 2 has been officially released upstream, we can hopefully see it soon in Debian.

I think you have to compile this part of pivy with the SConstruct script... But I never tested this, and it seems also not to be py3-compatible.

[kkremitzki]

Update post time!

I'd like to start my post by just copy+pasting some of the lintian warnings about the FreeCAD package:

Code: Select all

I: freecad: arch-dep-package-has-big-usr-share 22778kB 16%
N:
N:    The package has a significant amount of architecture-independent data
N:    (over 4MB, or over 2MB and more than 50% of the package) in /usr/share
N:    but is an architecture-dependent package. This is wasteful of mirror
N:    space and bandwidth since it means distributing multiple copies of this
N:    data, one for each architecture.
N:
N:    If the data in /usr/share is not architecture-independent, this is a
N:    Policy violation that should be fixed by moving the data elsewhere
N:    (usually /usr/lib).
N:
N:    Refer to Debian Developer's Reference section 6.7.5
N:    (Architecture-independent data) for details.
N:
N:    Severity: wishlist, Certainty: certain
N:
N:    Check: huge-usr-share, Type: binary
N:

Code: Select all

P: freecad: image-file-in-usr-lib usr/lib/freecad/Mod/TechDraw/TDTest/TestImage.png
N:
N:    This package installs a pixmap or a bitmap within /usr/lib. According to
N:    the Filesystem Hierarchy Standard, architecture-independent files should
N:    be placed within /usr/share instead.
N:
N:    Severity: pedantic, Certainty: certain
N:
N:    Check: files, Type: binary, udeb
N:

Code: Select all

I: freecad: font-in-non-font-package usr/share/freecad/Mod/TechDraw/Resources/fonts/osifont-lgpl3fe.ttf
N:
N:    This package contains a *.ttf, *.otf, or *.pfb file, file extensions
N:    used by TrueType, OpenType, or Type 1 fonts, but the package does not
N:    appear to be a dedicated font package. Dedicated font package names
N:    should begin with fonts-. (Type 1 fonts are also allowed in packages
N:    starting with xfonts-.) If the font is already packaged, you should
N:    depend on that package instead. Otherwise, normally the font should be
N:    packaged separately, since fonts are usually useful outside of the
N:    package that embeds them.
N:
N:    Severity: wishlist, Certainty: possible
N:
N:    Check: files, Type: binary, udeb
N:

Code: Select all

P: freecad source: debian-watch-does-not-check-gpg-signature
N:
N:    This watch file does not include a means to verify the upstream tarball
N:    using cryptographic signature.
N:
N:    If upstream distributions provide such signatures, please use the
N:    pgpsigurlmangle options in this watch file's opts= to generate the URL
N:    of an upstream GPG signature. This signature is automatically downloaded
N:    and verified against a keyring stored in
N:    debian/upstream/signing-key.asc.
N:
N:    Of course, not all upstreams provide such signatures, but you could
N:    request them as a way of verifying that no third party has modified the
N:    code against their wishes after the release. Projects such as
N:    phpmyadmin, unrealircd, and proftpd have suffered from this kind of
N:    attack.
N:
N:    Refer to the uscan(1) manual page for details.
N:
N:    Severity: pedantic, Certainty: certain
N:
N:    Check: watch-file, Type: source
N:

These are a few of the low-hanging fruits which would improve the package, especially the first two.

Having gotten more experienced about some of the things you can do with Debian packaging, I think I'd like to address them by beginning the process of splitting up the freecad package. This will involve, at first, splitting it into freecad, libfreecad-0.17, and freecad-common.

However, my plan involves going a little further. For the 0.18 release, we'll probably want to provide both Python 2 and Python 3 builds, and the best way IMO to provide both of these is using the DebianAlternatives system:

The Debian alternatives system creates a way for several programs that fulfill the same or similar functions to be listed as alternative implementations that are installed simultaneously but with one particular implementation designated as the default. For example many systems have several text editors installed at the same time. The vi program is a classic example of an editor that has many implementations such as nvi, elvis, vim, etc. but which one should be designated as the default?

Debian's alternatives system aims to solve the problem of selecting a default preferred version. Management is done through the update-alternatives program that creates, removes, maintains and displays information about the symbolic links constituting the Debian alternatives system. Priorities are assigned by package creators. The alternative with the highest priority determines the default value when in automatic mode. Additionally the local administrator may override the automatic selection with a manual selection.

This means that the freecad package will become a meta-package, and the Python 2/3 builds will be their own respective packages, freecad-python2 and freecad-python3, with /usr/bin/freecad being a symlink to one or the other, managed by update-alternatives.

Another advantage of doing this is that we can provide a freecad-micro package which is another /usr/bin/freecad alternative.

Besides working on that, I've also begun fixing the problem of the freecad-doc package, which was removed from Debian for not being Debian Free Software Guidelines compliant. The reason was that we were providing it as a compiled file, but we need to provide the original files. One of our "cousin packages", KiCAD, tracks their kicad-doc package in a separate repository, which is my plan. However, the docs built from the FreeCAD source also need to be packaged, so those will come from the original repo and be built as 'freecad-api-doc'.

As an aside, I have also been working on re-packaging Elmer for Debian (it used to be in the repos but hasn't for a long while.) Although it wasn't originally in the plan, the skills I've gotten have made it not much effort to work on it in between builds of other packages. I've also been working on updating the OpenFOAM package for OpenFOAM 5. (But these are just side efforts since I am blocked by build times on the main packages frequently, so doing things in parallel is easy.)

I'd also like to start packaging some of the FreeCAD extensions, since there was a request: RFP: freecad-extras-* -- additional macros and workbenches for FreeCAD -- suggestions for priority, anyone?

For the third lintian warning, I will package the osifont package referred to there.

For the fourth, we could start signing the packages, but that's something I'd like to run by others.

[NormandC]

Hi Kurt, quite a project! Seems like its scope widens every week

I think I'd like to address them by beginning the process of splitting up the freecad package. This will involve, at first, splitting it into freecad, libfreecad-0.17, and freecad-common.

My first reaction was to find it odd to create a libfreecad package, but then I looked at how Gimp is packaged, and it does have a libgimp package. I guess it makes sense.

But I would like for it to go even further, we've discussed this on the forum years ago. What wmayer and I discussed was to separate some workbenches into additional packages. For example, I never use FEM, yet FreeCAD installs a bunch of libraries (VTK comes to mind) that I will never need.

The problem is deciding which workbenches are to be considered "core" and which to be considered extras. Nobody would agree on this...

FYI a report was created for this years ago. issue #857

I'd also like to start packaging some of the FreeCAD extensions, since there was a request: RFP: freecad-extras-* -- additional macros and workbenches for FreeCAD -- suggestions for priority, anyone?

LOL - that report is inspired by the freecad community PPA I created.

Quite honestly, now that there is an addon manager in FreeCAD 0.17, is that whole idea of freecad-extras useful anymore? At the time, installing add-ons was a hassle, hence the PPA. But now, nobody is updating the freecad-extras PPA anymore. I really don't see the point.

On the other hand, the good thing with packages is that they are installed in the system folders rather than in the user's home. Therefore, they are available to all users on a PC.

But, one more item on the "against" column: the two most widely used add-ons mentioned in the Debian RFP, assembly2 and DrawingDimensioning, are no longer maintained since R-Frank passed away. One of them is not fully compatible with FC 0.17. As long as they are missing maintainers, it does not make much sense to package them...

[sgrogan]

LOL - that report is inspired by the freecad community PPA I created.

Quite honestly, now that there is an addon manager in FreeCAD 0.17, is that whole idea of freecad-extras useful anymore? At the time, installing add-ons was a hassle, hence the PPA. But now, nobody is updating the freecad-extras PPA anymore. I really don't see the point.

On the other hand, the good thing with packages is that they are installed in the system folders rather than in the user's home. Therefore, they are available to all users on a PC.

The addons-manager does a good job for pure python, and any effort, IMHO, would be better spent handling dependencies there. It is more difficult to handle the dependencies, but it would be more cross-platform.

But I would like for it to go even further, we've discussed this on the forum years ago. What wmayer and I discussed was to separate some workbenches into additional packages. For example, I never use FEM, yet FreeCAD installs a bunch of libraries (VTK comes to mind) that I will never need.

I continue to support this. The "drawback" is that the various FreeCAD packages would all have cross-dependencies to each other like OCCT.