packaging netgen6.2 for ubuntu

Having trouble installing or compiling FreeCAD? Get help here.
User avatar
looo
Posts: 3286
Joined: Mon Nov 11, 2013 5:29 pm

packaging netgen6.2 for ubuntu

Postby looo » Tue Sep 26, 2017 9:44 am

Finally I would like to give packaging netgen for ubuntu a try. But I fear some help from you guys is absolutely necessary to get through the debian packaging pain.
The first step is again:
https://forum.freecadweb.org/viewtopic. ... 20#p177784

So pulling the sources:

Code: Select all

mkdir netgen
cd netgen
apt-get source netgen
Now we need to change from autotools to CMake, upadate to occt7.2.0, ...
User avatar
sgrogan
Posts: 5798
Joined: Wed Oct 22, 2014 5:02 pm

Re: packaging netgen6.2 for ubuntu

Postby sgrogan » Tue Sep 26, 2017 10:02 pm

looo wrote:
Tue Sep 26, 2017 9:44 am
Now we need to change from autotools to CMake, upadate to occt7.2.0, ...
Wow looo, very ambitious :)
netgen is really a collection of packages

Code: Select all

Built packages (from the FreeCAD PPA)

    libnglib-4.9.13 Automatic 3d tetrahedral mesh generator shared library

    libnglib-dev Automatic 3d tetrahedral mesh generator development files

    netgen Automatic 3d tetrahedral mesh generator

    netgen-doc Automatic 3d tetrahedral mesh generator documentation

    netgen-headers Automatic 3d tetrahedral mesh generator internal headers
Unless you want to go through the exercise of doing everything with occt7.1.0 and then upgrading to occt7.2.0, then we need to update occt on the PPA first.

I'm extremely limited with internet bandwidth at the moment, computer is tethered to the phone with limited data, can you post a 7zip version of just the debian folder?

Also a list CMake flags needed to build nglib locally would help.
User avatar
looo
Posts: 3286
Joined: Mon Nov 11, 2013 5:29 pm

Re: packaging netgen6.2 for ubuntu

Postby looo » Wed Sep 27, 2017 7:29 am

sgrogan wrote:Unless you want to go through the exercise of doing everything with occt7.1.0 and then upgrading to occt7.2.0, then we need to update occt on the PPA first.
I think moving to occt7.2.0 is not a big problem, so I will use the available libraries...
sgrogan wrote:Also a list CMake flags needed to build nglib locally would help.
This is a script which compiles netgen with a virgin ubuntu16.04 from docker:

Code: Select all

sudo docker run -i -t --name xenial ubuntu:xenial

Code: Select all

apt-get update
apt-get install software-properties-common build-essential cmake zlib1g-dev python3-dev git tk-dev tcl-dev -y
add-apt-repository ppa:freecad-maintainers/freecad-daily -y
apt-get update
apt-get install libopencascade-data-exchange-dev \
                libopencascade-foundation-dev \
                libopencascade-modeling-algorithms-dev \
                libopencascade-modeling-data-dev -y
rm /usr/bin/python
ln -s /usr/bin/python3 /usr/bin/python
git clone https://github.com/pybind/pybind11
cp -r pybind11/include/pybind11 /usr/include/python3.5/
git clone https://github.com/looooo/netgen
cd netgen
git checkout 6.2_rebased5
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release \
      -DUSE_OCC=ON \
      -DUSE_PYTHON=ON \
      -DUSE_GUI=ON \
      -DUSE_SUPERBUILD=OFF \
      -DNG_INSTALL_DIR=/usr/lib/netgen \
      ..
make -j3
cpack .
As you can see at the end cpack is used to create a deb. But I don't know if this is really useful for us, as I have no idea if launchpad can work with cpack.
User avatar
looo
Posts: 3286
Joined: Mon Nov 11, 2013 5:29 pm

Re: packaging netgen6.2 for ubuntu

Postby looo » Wed Sep 27, 2017 8:58 am

ubuntu 14.04 is a bit more tricky...:

Code: Select all

sudo docker run -i -t --name trusty ubuntu:trusty

Code: Select all

apt-get update
apt-get install software-properties-common build-essential cmake zlib1g-dev python3-dev git tk-dev tcl-dev -y
add-apt-repository ppa:freecad-maintainers/freecad-daily -y
apt-get update
apt-get install libopencascade-data-exchange-dev \
                libopencascade-foundation-dev \
                libopencascade-modeling-algorithms-dev \
                libopencascade-modeling-data-dev -y
rm /usr/bin/python
ln -s /usr/bin/python3 /usr/bin/python
git clone https://github.com/pybind/pybind11
cp -r /pybind11/include/pybind11 /usr/include/python3.4/
git clone https://github.com/looooo/netgen
cd netgen
git checkout 6.2_rebased5
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release \
      -DUSE_OCC=ON \
      -DUSE_PYTHON=ON \
      -DUSE_GUI=ON \
      -DUSE_SUPERBUILD=OFF \
      -DNG_INSTALL_DIR=/usr/lib/netgen \
      -DPYTHON_INCLUDE_DIR=/usr/include/python3.4/ \
      -DPYTHON_INCLUDE_DIR2=/usr/include/python3.4/ \
      ..
make -j3
cpack .
But the problem is the installation of the deb:

Code: Select all

dpkg: dependency problems prevent configuration of netgen:
 netgen depends on liboce-ocaf-dev;
( Maybe we should rework the dependencies set by cpack to match occt7.1.0...
from netgen CMakeLists.txt:

Code: Select all

        if(USE_OCC)
            set(CPACK_DEBIAN_PACKAGE_DEPENDS "${CPACK_DEBIAN_PACKAGE_DEPENDS}, liboce-ocaf-dev")
        endif(USE_OCC)
I think using cpack with ubuntu running as a docker container would be the simplest way to generate the deb packages. Would it be possible to upload these manually created packages to the ppa?
User avatar
looo
Posts: 3286
Joined: Mon Nov 11, 2013 5:29 pm

Re: packaging netgen6.2 for ubuntu

Postby looo » Wed Sep 27, 2017 11:30 am

I uploaded packages for 16_04 and 14_04: https://github.com/looooo/netgen/releas ... ._rebased5
In my eyes manually compilation makes the most sense. With the docker script this also reproduceable, if we need to package netgen again...
gdo35
Posts: 189
Joined: Wed Jan 25, 2012 7:25 pm

Re: packaging netgen6.2 for ubuntu

Postby gdo35 » Wed Sep 27, 2017 8:04 pm

Hi loo,

I think container is not the most suitable way to build package.
Even if I am confident about it, we cannot guarantee that the netgen dependencies are not already installed in the docker image.

There is a tool dedicated to build packages in a clean packaging environment : pbuilder

I packaged netgen-5.3.1 with it, check here.

I was away for months so maybe I miss the need of netgen 6.2 but we could work together to package it if this version is now needed. ;)

++
gdo
User avatar
looo
Posts: 3286
Joined: Mon Nov 11, 2013 5:29 pm

Re: packaging netgen6.2 for ubuntu

Postby looo » Wed Sep 27, 2017 8:36 pm

gdo35 wrote:Even if I am confident about it, we cannot guarantee that the netgen dependencies are not already installed in the docker image.
I think the docker image is a subset of any ubuntu... So any ubuntu should contain at least the dependencies which are included in the docker image. But I am only guessing...
Anyway I started to work on the ppa-build and the difficult stuff is allready done. Build stops at this error: https://launchpadlibrarian.net/33891648 ... ING.txt.gz

Code: Select all

mkdir build_dir;cd build_dir;
cmake	-DCMAKE_BUILD_TYPE=Release \
		 	-DUSE_OCC=ON \
			-DUSE_PYTHON=ON \
			-DUSE_GUI=ON \
			-DUSE_SUPERBUILD=OFF \
			-DNG_INSTALL_DIR=/usr/lib/netgen/ \
			-DNG_INSTALL_DIR_INCLUDE=/usr/include/netgen/ \
			-DPYTHON_INCLUDE_DIR=/usr/include/python3.4/ \
			-DPYTHON_INCLUDE_DIR2=/usr/include/python3.4/ \
			-DPYTHON_LIBRARY=/usr/lib/python3.4/config-3.4m-x86_64-linux-gnu/libpython3.4.so \
			..
CMake Error: The source directory "/<<BUILDDIR>>" does not appear to contain CMakeLists.txt.
gdo35 wrote:I was away for months so maybe I miss the need of netgen 6.2 but we could work together to package it if this version is now needed.
no there is no need to package 6.2, but cmake simplifies the build quite a bit...
gdo35
Posts: 189
Joined: Wed Jan 25, 2012 7:25 pm

Re: packaging netgen6.2 for ubuntu

Postby gdo35 » Wed Sep 27, 2017 10:10 pm

looo wrote:
Wed Sep 27, 2017 8:36 pm
I think the docker image is a subset of any ubuntu... So any ubuntu should contain at least the dependencies which are included in the docker image. But I am only guessing...
It is a trap ! ;)
To create a minimal virgin rootfs for ubuntu or other debian like distro, you have to use a tool called debootstrap.
pbuilder uses debootstrap.

But even if your ubuntu docker image comes from debootstrap, there can be some tweaking we could not want, as you can see here, for example :
https://github.com/tianon/docker-brew-u ... Dockerfile

So the most safe and recommended way is to use pbuilder.
I let you read more about it : https://wiki.ubuntu.com/PbuilderHowto

Moreover, using pbuilder instead of directly test on launchpad avoids overloading servers with debug that can be done locally
looo wrote:
Wed Sep 27, 2017 8:36 pm
no there is no need to package 6.2, but cmake simplifies the build quite a bit...
+1

But... packaging binaries with cpack will be useless as only source packages can be uploaded on launchpad. (it answers a previous question you asked).
I do not know enough cpack but I guess it should generate the sources package that we could upload. Anyway we would have to maintain the debian folder.

5.3.1 is the last release on SourceForge. Is 6.2 pre-release ? (SourceForge git repo is down so I cannot check...).
User avatar
sgrogan
Posts: 5798
Joined: Wed Oct 22, 2014 5:02 pm

Re: packaging netgen6.2 for ubuntu

Postby sgrogan » Wed Sep 27, 2017 10:22 pm

looo wrote:
Wed Sep 27, 2017 8:58 am
Would it be possible to upload these manually created packages to the ppa?
I don't think that binary debs can be uploaded to Launchpad. Launchpad only accepts a "Debianized" source package and then builds and publishes that package. There is a loop hole that pre-compiled binaries can be included in the source package. So in practice the Launchpad "build" could just copy files. This is not the ideal method. We will need a binary deb somewhere on Launchpad to link FreeCAD against it for the PPA.
looo wrote:
Wed Sep 27, 2017 8:36 pm
Anyway I started to work on the ppa-build and the difficult stuff is already done. Build stops at this error: https://launchpadlibrarian.net/33891648 ... ING.txt.gz
Can you post a link to the debian folder you are using? For FreeCAD on the PPA we do it like this: https://git.launchpad.net/~freecad-main ... build-occt
In particular this line

Code: Select all

%:
	dh $@ --buildsystem=cmake --parallel --with python2
and then use extra flags. Ideally everything is done in the rules file and no scripts are necessary. But maybe it's not possible here.
User avatar
looo
Posts: 3286
Joined: Mon Nov 11, 2013 5:29 pm

Re: packaging netgen6.2 for ubuntu

Postby looo » Thu Sep 28, 2017 9:04 am

gdo35 wrote:It is a trap ! ;)
To create a minimal virgin rootfs for ubuntu or other debian like distro, you have to use a tool called debootstrap.
pbuilder uses debootstrap.
ok I understand ;)
gdo35 wrote: But... packaging binaries with cpack will be useless as only source packages can be uploaded on launchpad. (it answers a previous question you asked).
I do not know enough cpack but I guess it should generate the sources package that we could upload. Anyway we would have to maintain the debian folder.
sgrogan wrote:Can you post a link to the debian folder you are using?
I have come to the same conclusion. So I have added a debian directory with 4 files: https://github.com/looooo/netgen/tree/6 ... ed5/debian
gdo35 wrote:5.3.1 is the last release on SourceForge. Is 6.2 pre-release ? (SourceForge git repo is down so I cannot check...).
https://ngsolve.org/news/releases I don't know if it is officially released. But patching older stuff is for sure the not so good option, as it makes sending patches to upstream a lot more difficult...
sgrogan wrote:I don't think that binary debs can be uploaded to Launchpad. Launchpad only accepts a "Debianized" source package and then builds and publishes that package. There is a loop hole that pre-compiled binaries can be included in the source package. So in practice the Launchpad "build" could just copy files. This is not the ideal method. We will need a binary deb somewhere on Launchpad to link FreeCAD against it for the PPA.
Yes, I think we had this discussion already some time ago with the pivy package... It would have been a nice solution as cpack is really simple to use...
gdo35 wrote:Moreover, using pbuilder instead of directly test on launchpad avoids overloading servers with debug that can be done locally
For testing I stay with debuild for now. I think dependencies are already working. The problem is the build-script... Also having the failed build-logs online isn't a bad idea as we can all try to guess what is wrong with the builds...

the build is now working, but the install complaines:
https://launchpadlibrarian.net/33898867 ... ING.txt.gz

Code: Select all

cd build_dir; cmake -P cmake_install.cmake
-- Install configuration: "Release"
-- Installing: /<<PKGBUILDDIR>>/debian/tmp/usr/lib/cmake/netgen/netgen-targets.cmake
-- Installing: /<<PKGBUILDDIR>>/debian/tmp/usr/lib/cmake/netgen/netgen-targets-release.cmake
-- Installing: /<<PKGBUILDDIR>>/debian/tmp/usr/lib/cmake/netgen/NetgenConfig.cmake
CMake Error at libsrc/general/cmake_install.cmake:44 (FILE):
  file cannot create directory: /usr/include/netgen.  Maybe need
  administrative privileges.
Call Stack (most recent call first):
  libsrc/cmake_install.cmake:37 (INCLUDE)
  cmake_install.cmake:60 (INCLUDE)