Linux package maintainers: How to improve maintenance of FreeCAD packages?

Having trouble installing or compiling FreeCAD? Get help here.
simonvanderveldt
Posts: 61
Joined: Tue Mar 14, 2017 2:11 pm

Linux package maintainers: How to improve maintenance of FreeCAD packages?

Postby simonvanderveldt » Tue Mar 28, 2017 7:00 pm

This post is mainly aimed at everyone who maintains a (Linux) package for FreeCAD.

I recently started fixing open issues with Gentoo's FreeCAD ebuild for git master, which means doing similar things like @NormandC has done for Ubuntu.

Whilst doing so I ran into a relatively small issue with files being installed in the wrong locations. Trying to start with something small I looked at how to fix the installation of FreeCAD's icons so they end up in the correct location. PR can be found here https://github.com/FreeCAD/FreeCAD/pull/617

There was some feedback from @wwmayer about responsibilities. This would be the thing I'd like to get some feedback on from other package maintainers.
Basically there would be two options:
- FreeCAD's build/install script installs the icons and other related items like freecad.desktop and freecad.sharedmimeinfo.
If so, then these files should be installed to the correct locations, which means they should be OS (Linux, *BSD, Mac, Windows) specific
- FreeCAD's build/install script doesn't install the icons and other related items like freecad.desktop and freecad.sharedmimeinfo. This will be the task of the packagers.
If so it should not do anything with these resources during install.

I guess the same question is relevant for other resources like like documentation and examples as well.

Also I have many other questions and suggestions, but I'd like to start with this small one. More will probably follow :)

What do other package maintainers think of this?
simonvanderveldt
Posts: 61
Joined: Tue Mar 14, 2017 2:11 pm

Re: Linux package maintainers: How to improve maintenance of FreeCAD packages?

Postby simonvanderveldt » Tue Mar 28, 2017 7:15 pm

luzpaz pointed me at the fact that one can abuse the quoting mechanism to send notifications, so doing so to notify the people I know are doing something related to maintaining packages :P

NormandC wrote:Ubuntu packages

m42kus wrote:Arch PKGBUILDs

cox wrote:Arch PKGBUILDs
Bluewind
Posts: 7
Joined: Tue Jan 17, 2017 9:00 am

Re: Linux package maintainers: How to improve maintenance of FreeCAD packages?

Postby Bluewind » Wed Mar 29, 2017 10:36 pm

I've recently got a report about an icon error (https://bugs.archlinux.org/task/53320) which brought me to this thread. FWIW I think installing freecad to match the Arch Linux' directory layout works rather well. One gripe I have is that I currently have to work around the mod dir being installed and expected at an incorrect location. I'm not sure why that is not part of DATADIR or why there is no dedicated cmake setting for this directory. If there were dedicated settings and the correct values were used by the code, the bug I linked would also go away.

As for the two options: I much prefer if upstream handles installing all files. That way I don't have to worry about potentially missing files because I didn't see a new addition. If the files are installed to incorrect locations or if you install files that I do not want in the package, I can easily spot those and move/delete them. If I have to keep track of which additional files are shipped in the tarball and which ones I have to install that makes my job a lot more difficult for little gain.

If you have files like the icons which may require different paths or may not be applicable to different operating systems, I'd say that you simply do not set a default path and instead throw an error telling the packager to either set the path or disable installation of the files via some other option. I don't know if this kind of behaviour is possible with cmake though.

All that being said, the Arch Linux package is still at 0.16 since that is the latest release. I don't know if any of those problems have since been resolved in master, but given there is a thread about them I think they haven't.
User avatar
sgrogan
Posts: 3060
Joined: Wed Oct 22, 2014 5:02 pm

Re: Linux package maintainers: How to improve maintenance of FreeCAD packages?

Postby sgrogan » Wed Mar 29, 2017 11:00 pm

simonvanderveldt wrote:- FreeCAD's build/install script installs the icons and other related items like freecad.desktop and freecad.sharedmimeinfo.
If so, then these files should be installed to the correct locations, which means they should be OS (Linux, *BSD, Mac, Windows) specific
- FreeCAD's build/install script doesn't install the icons and other related items like freecad.desktop and freecad.sharedmimeinfo. This will be the task of the packagers.
If so it should not do anything with these resources during install.

Maybe a CMake Boolean? I would hesitate to change the default behavior that works for some, because of patches or not. Perhaps an option "No Thanks, I'll do it myself"?
User avatar
NormandC
Posts: 12279
Joined: Sat Feb 06, 2010 9:52 pm
Location: Québec, Canada

Re: Linux package maintainers: How to improve maintenance of FreeCAD packages?

Postby NormandC » Thu Mar 30, 2017 2:37 am

Hello Simon,

To be honest, I've always focused on the mechanics of Debian packaging, and I was mostly starting from an existing package in the Debian/Ubuntu repositories as reference. I'm not well versed in the Debian Policy, I just know enough to get by. And I have a very limited understanding of CMake or software compilation.

But I agree on the general view that FreeCAD should aim to install files in standard locations, if such universal standards exist across Linux distributions.
wmayer
Site Admin
Posts: 11352
Joined: Thu Feb 19, 2009 10:32 am

Re: Linux package maintainers: How to improve maintenance of FreeCAD packages?

Postby wmayer » Thu Mar 30, 2017 8:33 am

Bluewind wrote:I've recently got a report about an icon error (https://bugs.archlinux.org/task/53320) which brought me to this thread.

I don't know what exactly the problem is. I assume inside /usr/bin you have the executable freecad or FreeCAD. What is it, the executable itself or a symlink? For the former case it would explain the behaviour because relative to the path of the executable it computes the path where it expects the Mod directory.

Bluewind wrote:One gripe I have is that I currently have to work around the mod dir being installed and expected at an incorrect location.

Why do you consider /usr/lib/freecad/Mod as incorrect location? Or does Arch Linux put the files somewhere else?

It was me who did the initial work on "debianizing" FreeCAD and at that time I checked other big packages where their files are getting installed to. As reference I oriented me on openoffice which also puts its binary files in a sub-directory underneath /usr/lib/openoffice and AFAIR according to freedesktop.org this is allowed.

I'm not sure why that is not part of DATADIR or why there is no dedicated cmake setting for this directory.

Isn't it disallowed to put any architecture-dependent files into DATADIR?

As for the two options: I much prefer if upstream handles installing all files. That way I don't have to worry about potentially missing files because I didn't see a new addition.

From the beginning on FreeCAD was designed to run on all major platforms (Windows, Linux, macOS, ...) but IMO the cmake scripts shouldn't try to handle all platform specific cases (on installation) because this makes them complicated and nearly unmaintainable. I see this in the responsibility of the package maintainers because they know best where files have to go to. And it also depends on the Linux distribution what is considered the standard directory. When I was working with SuSE (> 16 years ago) many software packages have been installed to /opt, not /usr.

All what FreeCAD can do is to offer the relevant (cmake) switches to configure it to put files in the expected directories.
Bluewind
Posts: 7
Joined: Tue Jan 17, 2017 9:00 am

Re: Linux package maintainers: How to improve maintenance of FreeCAD packages?

Postby Bluewind » Thu Mar 30, 2017 9:23 am

wmayer wrote:I don't know what exactly the problem is. I assume inside /usr/bin you have the executable freecad or FreeCAD. What is it, the executable itself or a symlink? For the former case it would explain the behaviour because relative to the path of the executable it computes the path where it expects the Mod directory.


The executable is in /usr/bin so that path detection explains the issue indeed. I have to wonder though, why do you even detect the path like this? Wouldn't it be better to have a cmake setting and use the value of that instead?

Why do you consider /usr/lib/freecad/Mod as incorrect location? Or does Arch Linux put the files somewhere else?


/usr/lib/freecad/Mod would actually be perfect, but it looks like the Mod files are installed to CMAKE_INSTALL_PREFIX/Mod. You can see the commands we use to build the package here: https://git.archlinux.org/svntogit/comm ... es/freecad
Am I missing a cmake setting or something?

You also mentioned that DATADIR shouldn't contain architecture-specific files. That is entirely correct and it appears that I have missed this when taking over the package. I see some python files in there and we generally install those to /usr/lib too because they are normally also compiled to .pyo. I'm not sure why I don't see those here.

If you want to see the file list of our package you can do so here: https://www.archlinux.org/packages/comm ... cad/files/

Ideally I'd like the following layout:
/usr/bin: executables
/usr/lib/freecad: arch-dependent support files, libraries
/usr/share/freecad: arch-independent files, documentation, images
/usr/share/applications: .desktop files
/usr/share/mime/packages: MIME type mapping
/usr/share/icons/hicolor: icons for .desktop file

All what FreeCAD can do is to offer the relevant (cmake) switches to configure it to put files in the expected directories.


That's all I'm asking for.

Currently I see CMAKE_INSTALL_{DATADIR,DOCDIR,INCLUDEDIR,LIBDIR} in cmake. I'd like to see BINDIR being added and the Mod directory be put somewhere other than PREFIX/Mod, possibly into LIBDIR. If possible I'd also like settings for the .desktop files, MIME mappings and icons, but I can live without those if you consider them too specific.
wmayer
Site Admin
Posts: 11352
Joined: Thu Feb 19, 2009 10:32 am

Re: Linux package maintainers: How to improve maintenance of FreeCAD packages?

Postby wmayer » Thu Mar 30, 2017 12:44 pm

I have to wonder though, why do you even detect the path like this? Wouldn't it be better to have a cmake setting and use the value of that instead?

Actually there so far was never a problem expecting the Mod directory to be sibling of the executable's parent directory.

/usr/lib/freecad/Mod would actually be perfect, but it looks like the Mod files are installed to CMAKE_INSTALL_PREFIX/Mod. You can see the commands we use to build the package here:

Then instead of

Code: Select all

  cmake \
    -DCMAKE_BUILD_TYPE=Release \
    -DCMAKE_INSTALL_PREFIX:PATH="/usr" \
    -DCMAKE_INSTALL_DOCDIR:PATH="share/freecad/doc" \
    -DCMAKE_INSTALL_DATADIR:PATH="share/freecad" \
    -DCMAKE_INSTALL_LIBDIR:PATH="lib/freecad" \
    -DOCC_INCLUDE_DIR:PATH=/opt/opencascade/inc/ \
    -DPYTHON_EXECUTABLE:FILEPATH=/usr/bin/python2 \
    -DFREECAD_USE_EXTERNAL_PIVY:BOOL=ON \
    -DBUILD_START=OFF \
    -DBUILD_WEB=OFF


try this:

Code: Select all

  cmake \
    -DCMAKE_BUILD_TYPE=Release \
    -DCMAKE_INSTALL_PREFIX:PATH="/usr/lib/freecad" \
    -DCMAKE_INSTALL_DOCDIR:PATH="/usr/share/freecad/doc" \
    -DCMAKE_INSTALL_DATADIR:PATH="/usr/share/freecad" \
    -DOCC_INCLUDE_DIR:PATH=/opt/opencascade/inc/ \
    -DPYTHON_EXECUTABLE:FILEPATH=/usr/bin/python2 \
    -DFREECAD_USE_EXTERNAL_PIVY:BOOL=ON \
    -DBUILD_START=OFF \
    -DBUILD_WEB=OFF

However, according to Simon doing it this way wouldn't be 100% correct. To go conform how things worked with automake we can introduce the variable datarootdir and then the cmake arguments can be reduced to something like this:

Code: Select all

  cmake \
    -DCMAKE_BUILD_TYPE=Release \
    -DCMAKE_INSTALL_PREFIX:PATH="/usr/lib/freecad" \
    -DCMAKE_INSTALL_DATAROOTDIR:PATH="/usr/share/freecad" \
    -DOCC_INCLUDE_DIR:PATH=/opt/opencascade/inc/ \
    -DPYTHON_EXECUTABLE:FILEPATH=/usr/bin/python2 \
    -DFREECAD_USE_EXTERNAL_PIVY:BOOL=ON \
    -DBUILD_START=OFF \
    -DBUILD_WEB=OFF


Currently I see CMAKE_INSTALL_{DATADIR,DOCDIR,INCLUDEDIR,LIBDIR} in cmake. I'd like to see BINDIR being added and the Mod directory be put somewhere other than PREFIX/Mod, possibly into LIBDIR.

I see.

If possible I'd also like settings for the .desktop files, MIME mappings and icons, but I can live without those if you consider them too specific.

Yes, that's for sure too platform specific.
Bluewind
Posts: 7
Joined: Tue Jan 17, 2017 9:00 am

Re: Linux package maintainers: How to improve maintenance of FreeCAD packages?

Postby Bluewind » Thu Mar 30, 2017 6:32 pm

wmayer wrote:try this:

Code: Select all

  cmake \
    -DCMAKE_BUILD_TYPE=Release \
    -DCMAKE_INSTALL_PREFIX:PATH="/usr/lib/freecad" \
    -DCMAKE_INSTALL_DOCDIR:PATH="/usr/share/freecad/doc" \
    -DCMAKE_INSTALL_DATADIR:PATH="/usr/share/freecad" \
    -DOCC_INCLUDE_DIR:PATH=/opt/opencascade/inc/ \
    -DPYTHON_EXECUTABLE:FILEPATH=/usr/bin/python2 \
    -DFREECAD_USE_EXTERNAL_PIVY:BOOL=ON \
    -DBUILD_START=OFF \
    -DBUILD_WEB=OFF



I guess I won't get around symlinking the executable for now and apart from that, this works perfectly and also fixes the bug I got AFAICT. Thanks!
simonvanderveldt
Posts: 61
Joined: Tue Mar 14, 2017 2:11 pm

Re: Linux package maintainers: How to improve maintenance of FreeCAD packages?

Postby simonvanderveldt » Sun Apr 02, 2017 10:09 am

Thanks for the feedback everyone!
I don't mind to invest some time to try and fix some of the issues with the CMakeLists, but I would like to know up front if these changes would be accepted. It'll take a bit of time and I'd hate to see that wasted :)

My suggestions would be to do the following:
- Use GNUInstallDirs and make use of it's variables for the different types of files/directories
- Update the CmakeLists files so the INSTALL calls all reference one of these variables

This then frees CMAKE_INSTALL_PREFIX again to serve it's intended purpose of being a prefix for everything and allowing everything to be installed to any prefix directory by simply setting it, for example to /usr/local

This would mean a simple cmake with optionally passing in a prefix would be enough to configure/install FreeCAD on Linux, no need to pass all the extra directories, though you obviously still could if you wanted to.

I'll focus on the binaries, the arch-dependent files (libs) and arch-independent files (docs, resources) and will skip the other stuff for now.

Obviously I'll make sure it'll only affect Linux. Does anyone know if this would be relevant for BSD as well? Or is that not a relevant target at the moment?