[Ubuntu Daily PPA] Transitioning to OCCT7, VTK7...

Having trouble installing or compiling FreeCAD? Get help here.
User avatar
NormandC
Posts: 12030
Joined: Sat Feb 06, 2010 9:52 pm
Location: Québec, Canada

[Ubuntu Daily PPA] Transitioning to OCCT7, VTK7...

Postby NormandC » Sun Sep 18, 2016 7:36 am

This is a new PPA maintenance coordination thread with a narrower focus.

For the current 0.17 development cycle, MacOS X and Windows builds have been switched from OCE 0.17 (OCC 6.8.0) to OCCT 7.0.0 for a while now. Ubuntu Daily Builds have been left in the dust with the now obsolete OCE 0.17, and I'd like to help getting this situation remedied so that all platforms share the same OCC version.

I've started looking into creating an Open Cascade 7.0.0 debian package. It's a major version change, which means we cannot simply use and lightly edit the contents of the existing oce package debian folder. I once built a package for a major version change, and I remember I had to heavily edit all the *-dev.install files. I'm also certain that I wrote about it on the forum, Yorik even gave me some help, but I can't locate any relevant forum topic. So I'll have to relearn how to do it, and I'll be documenting the process as thoroughly as I can this time. Maybe keeping local notes as well. :roll:

After poking here and there on the opencascade.org website, I found this relevant section in the documentation about the changes brought by OCCT 7.0.0:

https://dev.opencascade.org/doc/overview/html/occt_dev_guides__upgrade.html#upgrade_700

Relevant bits:
Legacy persistence for shapes and OCAF data based on Storage_Schema (toolkits TKPShape, TKPLCAF, TKPCAF, TKShapeShcema, TLStdLSchema, TKStdSchema, and TKXCAFSchema) has been removed in OCCT 7.0.0.

These toolkits are referenced in liboce-ocaf10.install, liboce-ocaf-dev.install, liboce-ocaf-lite10.install and liboce-ocaf-lite-dev.install. They should be removed, and the lists of files the -dev.install files contain need to be regenerated.

Visualization CAF attributes have been moved into a new toolkit TKVCAF. If your application uses the classes from TPrsStd package then add link to TKVCAF library.

I'm guessing this new toolkit should be added to the visualization install files.


EDIT 2016/09/22: found this file listing modules for separate classes: adm/MODULES

Other points:

  • Open CASCADE now relies on VTK (this is not needed as FreeCAD uses VTK independently from OCC), FreeCAD does as well, we may need to create a VTK7 package first
  • The CMakeLists.txt file sets cmake 2.8.12 as the minimum required version, trusty barely complies with the same version. This means it won't build on Ubuntu 12.04 LTS, but I believe we stopped supporting it anyway.
  • Should we create a git repo on Launchpad for the debian packaging? I guess it would make sense.
  • Should we also clone the OCC git repo to Launchpad? Or simply put everything, the upstream sources and the debian folder into the same repo? Kind of like the Debian Science guys are doing: https://anonscm.debian.org/cgit/debian-science/packages/oce.git/
  • Regarding the package naming, I'm really not fond of how the liboce runtime packages get an arbitrary number bumped for each OCE release (liboce-foundation10, liboce-modeling10, etc.). I propose we add the real OCC version number to the package name: libopencascade7 (without minor version and maintenance version digits since they are zero). When OCC is updated to 7.1, we can name the new packages libopencascade7.1. Minor maintenance version digits (i.e. 7.0.1) should not change the package name, we'd just provide an updated package. But I doubt this will happen, OCCT does not seem to publish maintenance versions.
  • I seem to recall that vejmarie needed to patch netgen 5.1 to get it working. Is it still the case? (Found this patch from looo, what about wmayer's?)
  • What about VTK7? Does it need to be patched too?
  • Adding support for TBB parallel processing (libtbb2?) along with vejmarie's patch to improve read time on STEP files (topic: Re: STEP file read might be crazy long ...)
  • TBB might also be useful for Boolean operations: OCCT Forum / Current Progress - Parallelization of Boolean Operations Algorithm
@sgrogan, if you could post the links to the relevant topics about occ7/vtk7/netgen, that would be helpful, that is if you've bookmarked them. Otherwise I'll try to hunt for them.

That's it for now...
Last edited by NormandC on Fri Sep 23, 2016 3:07 am, edited 9 times in total.
looo
Posts: 672
Joined: Mon Nov 11, 2013 5:29 pm

Re: [Ubuntu Daily PPA] Transitioning to OCCT7, VTK7...

Postby looo » Sun Sep 18, 2016 4:53 pm

thanks for your efforts. This seems to be a monster task.

regarding netgen: I have contacted the netgen developer, and he will apply patches from vejmarie and wmayer to the netgen master branch. The patch for making FreeCAD work with netgen master is here: http://forum.freecadweb.org/viewtopic.php?f=17&t=17374 (but only tested from my side). Also there is another change within netgen necessary, but I haven't got the answer if this could be merged in netgen. So I would wait until these patches are in netgen, and then use a snapshot of master to create the package.

regarding vtk: why not wait for an official package. Or use this one: https://launchpad.net/~elvstone/+archive/ubuntu/vtk7
User avatar
NormandC
Posts: 12030
Joined: Sat Feb 06, 2010 9:52 pm
Location: Québec, Canada

Re: [Ubuntu Daily PPA] Transitioning to OCCT7, VTK7...

Postby NormandC » Sun Sep 18, 2016 5:17 pm

looo wrote:thanks for your efforts. This seems to be a monster task.

Not really, it should be straightforward for someone who knows they're doing... :roll: But that isn't me! All I've been doing for all those years is barely getting by. My knowledge of packaging remains limited, and not being a coder I have a limited understanding of the OCC structure.

looo wrote:regarding vtk: why not wait for an official package.

Because it will likely take forever. Vtk is now at 7.7, and Debian/Ubuntu are still stuck with 6.2! Packaging must be done by the Debian Science guys, then it will trickle down to Ubuntu which will yet take more time. There is currently no one working on it. http://forum.freecadweb.org/viewtopic.php?f=10&t=12534&start=170#p135069

I'll wager it won't be ready for the 0.17 release. But we should have it months before release to get enough time for testing.

Unless the FEM guys say that we can have full FEM functionality with 6.2? I would seriously prefer that.


Funny thing, that package was done by the guy who enquired about vtk7 status on the debian-science mailing list 7 months ago.

We couldn't copy his package because it's built against Qt5 and Python3. And I don't like that it's a single huge package, AFAIK the FEM guys only use a subset of vtk.

We may end up having to do the same (or simply recompile his package) if we can't figure out how to break it down into multiple packages. But if that happens, I will push to remove FEM from the core of FreeCAD and provide it as an additional package. I have no use for it, as I'm sure is the case for a large majority of FreeCAD users. The FEM module has brought huge dependencies, IMO FreeCAD is becoming bloatware. If there is no consensus to do that cross-platform, I will at least do it for the Ubuntu PPA.
Last edited by NormandC on Sun Sep 18, 2016 7:10 pm, edited 3 times in total.
User avatar
NormandC
Posts: 12030
Joined: Sat Feb 06, 2010 9:52 pm
Location: Québec, Canada

Re: [Ubuntu Daily PPA] Transitioning to OCCT7, VTK7...

Postby NormandC » Sun Sep 18, 2016 5:18 pm

looo wrote:So I would wait until these patches are in netgen, and then use a snapshot of master to create the package.

Thanks for your work on netgen. Do you know how much time that might take?
vejmarie
Posts: 441
Joined: Mon Jan 04, 2016 4:52 pm
Location: Somewhere between France, USA and Taiwan
Contact:

Re: [Ubuntu Daily PPA] Transitioning to OCCT7, VTK7...

Postby vejmarie » Sun Sep 18, 2016 8:05 pm

NormandC wrote:
looo wrote:So I would wait until these patches are in netgen, and then use a snapshot of master to create the package.

Thanks for your work on netgen. Do you know how much time that might take?


Netgent team is unfortunatly "slow" to integrate patches from a past experience but hopefully this has progressed ...
User avatar
NormandC
Posts: 12030
Joined: Sat Feb 06, 2010 9:52 pm
Location: Québec, Canada

Re: [Ubuntu Daily PPA] Transitioning to OCCT7, VTK7...

Postby NormandC » Sun Sep 18, 2016 8:13 pm

Well, as I replied on your "STEP import is slow" topic, we can apply patches to a debian package without modifying the source code. I believe that would be our best option for now, if we don't want to stay at the mercy of other projects' development pace.

This is actually quite simple, a "patches" folder is created inside the "debian" folder. When the package is built, the patch is applied to the untouched source tarball. Just dropping the patch file in the folder is not sufficient though: it needs to be created with a patching tool. I must have kept a link to the tutorial I used last time, I'll try to dig it up.
looo
Posts: 672
Joined: Mon Nov 11, 2013 5:29 pm

Re: [Ubuntu Daily PPA] Transitioning to OCCT7, VTK7...

Postby looo » Sun Sep 18, 2016 9:12 pm

Using patches is fine, but only works as long as there are no changes in the corresponding source... With conda it is possible to use git patches, which are created with "git format-patch ...". I did this with netgen, but after some changes made by the netgen developers the patches were no't working any more... So now I use a fork to create the conda netgen builds... Hopefully this dependency mess has an end soon.

Maybe some day we can find a way on how to provide c++ extensions without having the source directly inside of freecad. This way we could have for example an external FEM. (BUT, I know, not everybody is a fan of separating FreeCAD)

As long as the dependencies are in the current state, you can have a look at conda. I have now all the packages for latest freecad on the freecad-channel. It will only work on very recent linux distros (eg ubuntu 16.04) but I will work to get it running with older versions too. Creating packages with conda is really easy. Daily builds are possible with travis (see conda-forge on github).... But I think I have already talked too much about it.
User avatar
NormandC
Posts: 12030
Joined: Sat Feb 06, 2010 9:52 pm
Location: Québec, Canada

Re: [Ubuntu Daily PPA] Transitioning to OCCT7, VTK7...

Postby NormandC » Sun Sep 18, 2016 9:45 pm

looo wrote:Using patches is fine, but only works as long as there are no changes in the corresponding source...

On the PPA we don't change the dependencies on a whim, and only upload packages if the ones on the Ubuntu repositories are not suitable or are absent. Once a dependency package is uploaded, it will stay that way until we decide to update it, which might take a year or more. When that happens, we'll just have to see if the patches are still required...

looo wrote:As long as the dependencies are in the current state, you can have a look at conda. I have now all the packages for latest freecad on the freecad-channel. It will only work on very recent linux distros (eg ubuntu 16.04) but I will work to get it running with older versions too. Creating packages with conda is really easy.

Having gone through 4 years of Ubuntu PPA management, I have a great appreciation for the work you've done on Anaconda.

I've downloaded the freecad_build_env.yml build file, I'm seeing many dependencies I'm not familiar with... OK, that's just specific-conda packages I think.

BTW libgfortran is no longer required, have a look athe Daily Builds PPA dependencies.
https://git.launchpad.net/~freecad-maintainers/+git/gitpackaging/tree/debian/control?h=dailybuild
looo
Posts: 672
Joined: Mon Nov 11, 2013 5:29 pm

Re: [Ubuntu Daily PPA] Transitioning to OCCT7, VTK7...

Postby looo » Mon Sep 19, 2016 7:15 am

I've downloaded the freecad_build_env.yml build file, I'm seeing many dependencies I'm not familiar with... OK, that's just specific-conda packages I think.

OT: This was only a test, and actually it doesn't work right now with current conda... It is a nice idea to have such enviroment-snapshots. This way it is easy to provide newcomers with all resources to get into FreeCAD development. Simply create a new enviroment with conda env create freecad/fc_2_build clone/fork the freecad master and start compiling/ coding ...
User avatar
sgrogan
Posts: 2969
Joined: Wed Oct 22, 2014 5:02 pm

Re: [Ubuntu Daily PPA] Transitioning to OCCT7, VTK7...

Postby sgrogan » Mon Sep 19, 2016 4:51 pm

NormandC wrote:@sgrogan, if you could post the links to the relevant topics about occ7/vtk7/netgen, that would be helpful, that is if you've bookmarked them. Otherwise I'll try to hunt for them.

I will post a summary when I get home from work tonight.