GSoC 2018: FreeCAD Configuration Management Project

[kkremitzki]

The Configuration Management Project seems pretty interesting this year. Although I did GSoC last summer I am also eligible to apply this summer as well, although it really depends on how my job search goes pre/post graduation in May. If I do apply, it'll probably be for this one since it lines up well with my packaging work lately.

Maybe a new thread, I'd like to discuss this.

This interesting project idea is described in detail on the wiki. I'm considering applying to do this project and as suggested I'm creating this thread for discussing it!

I think my work in getting OCCT to Debian would segue well into this. The general gist of the project would be to make development & releases easier, especially on Linux (Debian-based) and Windows (to the extent that I can). Here are a few possible sub-components I've thought of, with varying "sizes":

• getting the Buildbot tools working on Linux & Windows VMs on vejmarie's servers -- this could mean automated delegated builds from trusted developers on Linux/Windows, so for example Assembly3 branch patched FreeCAD
• getting combined sub-project docs hosted thanks to vejmarie's servers
• getting PySide 2 into Debian to fast-track Qt 5 Python 3 FreeCAD development & usage
• simplifying Windows dev environment setup and getting debug builds of dependencies
• improving new dev docs for Linux & Windows based on work in other sub-components

There's lots of other things that could fall under this project, though. Thoughts?

[kkremitzki]

I wonder if it would be feasible to think about breaking out 3rd party components from FreeCAD under this project, e.g. salome.

[Kunda1]

This is an important idea that I hope gets attention in this GSOC. Thanks @kkremitzki
FYI, it's good to add a mentor section and write who the potential mentors will be for this GSOC project.

[kkremitzki]

As far as I know that would be

[sgrogan]

As far as I know that would be

I am willing to be a mentor , but don't know the process to be qualified/accepted, and I don't know if this needs to be done prior to a project proposal being accepted. We would need an additional mentor, I believe we need 2 per project now.

There's lots of other things that could fall under this project, though. Thoughts?

FreeCAD Dependency Graph
I think the first thing is to create a maintainable dependency graph for FreeCAD on Linux/windows (maybe OSX) platforms. And an easy way to update this. Maybe Graphviz? If we can easily pull information from this to keep the Wiki up to date would be hugely beneficial in and of itself.

For Ubuntu/Debian
We need an easier way to maintain packages on the PPA. Building the packages on Launchpad, using gitbuilder has helped, but we need an easier way to generate the Debian folder. Maybe this is just a documented procedure how to do this in a FreeCAD specific context. Your packaging work with upstream Debian is very valuable here. IMHO mirroring the dependent source repo on Launchpad and building there would ease maintenance. After all there will older LTS's that don't have the latest and greatest, and even the newest will become stale from a FreeCAD point of view before they reach EOL.

It would be nice to have an "Universal" Debian folder (i.e. one that works on Debian) in the FreeCAD source. We could simply use the one from the PPA, but it is Ubuntu specific. The existing one in FreeCAD source is not complete/up-to-date for any modern release.

For Windows
Conda is it. @looo has spent hundreds(thousands?) of hours getting us here. I'm able to reliably build release versions of FreeCAD using Conda as a package manager. For now I have to "install" FreeCAD into the conda environment to make it run. First step would be to strip out the runtime stuff from the conda environment to make the build portable without the conda overhead. This is not strictly required, but I think it is presenting a barrier that make some reluctant to adopt.

I am 100% convinced that all needed run and build time dependencies can be stripped out of conda to build a traditional Libpack. I'm 99% that it can be done with CMake as a conda package.

Documenation
This is always the hardest. I'm the most guilty, but great work get's done/discovered, and I spend many Duck Duck-Go cycles trying to remember what ended up working.

I know, a long post, but I can further elaborate on any point. This is a greater scope than a single GSoC project, so you can refine it as applicable.

[wandererfan]

I am willing to be a mentor , but don't know the process to be qualified/accepted, and I don't know if this needs to be done prior to a project proposal being accepted. We would need an additional mentor, I believe we need 2 per project now.

I am woefully under-qualified in this area but perfectly willing to be a mentor.

[kkremitzki]

As far as I know that would be

I am willing to be a mentor , but don't know the process to be qualified/accepted, and I don't know if this needs to be done prior to a project proposal being accepted. We would need an additional mentor, I believe we need 2 per project now.

There's lots of other things that could fall under this project, though. Thoughts?

FreeCAD Dependency Graph
I think the first thing is to create a maintainable dependency graph for FreeCAD on Linux/windows (maybe OSX) platforms. And an easy way to update this. Maybe Graphviz? If we can easily pull information from this to keep the Wiki up to date would be hugely beneficial in and of itself.

Sure, Graphviz is nice, or perhaps something with NetworkX which I've used before on a graph theory project.

For Ubuntu/Debian
We need an easier way to maintain packages on the PPA. Building the packages on Launchpad, using gitbuilder has helped, but we need an easier way to generate the Debian folder. Maybe this is just a documented procedure how to do this in a FreeCAD specific context. Your packaging work with upstream Debian is very valuable here. IMHO mirroring the dependent source repo on Launchpad and building there would ease maintenance. After all there will older LTS's that don't have the latest and greatest, and even the newest will become stale from a FreeCAD point of view before they reach EOL.

Right now because I'm still stuck putting extra polish on OCCT 7.2 for Debian, I haven't moved towards making that package available on the PPA because it's undergoing changes. Opencascade-draw is quite a bit of a challenge to get into an acceptable state, but it'll help make it easier to get bugs upstream. P.S. I'm trying to get a change OK'd to rename libopencascade-* -> libocct-*. I am already tired of typing such a long name.

But once that's stabilized it should mostly be a matter of changing the distribution in the changelog to make builds on other platforms, and maybe some bit of patching as well.

It would be nice to have an "Universal" Debian folder (i.e. one that works on Debian) in the FreeCAD source. We could simply use the one from the PPA, but it is Ubuntu specific. The existing one in FreeCAD source is not complete/up-to-date for any modern release.

I can't remember where I read it, but with Debian packaging now they don't recommend keeping the debian/ folder inside the upstream repo. Now that they are running a Gitlab instance, it would be appropriate for this to live inside the Debian "downstream" repository at https://salsa.debian.org/science-team/freecad.

For Windows
Conda is it. @looo has spent hundreds(thousands?) of hours getting us here. I'm able to reliably build release versions of FreeCAD using Conda as a package manager. For now I have to "install" FreeCAD into the conda environment to make it run. First step would be to strip out the runtime stuff from the conda environment to make the build portable without the conda overhead. This is not strictly required, but I think it is presenting a barrier that make some reluctant to adopt.

Yep, I think Conda is the way forward for distribution on Windows as well. Gotta work to make sure it's good on the development side of things as well. I wasn't very satisfied trying to set up a Windows dev VM and trying to Google the download links of some of the dev stuff required in the wiki... only to find it wasn't on Microsoft's website anymore. It'd be nice to have a standard Qt Creator + some non-MS compiler setup.

I am 100% convinced that all needed run and build time dependencies can be stripped out of conda to build a traditional Libpack. I'm 99% that it can be done with CMake as a conda package.

Documenation
This is always the hardest. I'm the most guilty, but great work get's done/discovered, and I spend many Duck Duck-Go cycles trying to remember what ended up working.

I think at the very least the dependency graph + as much of those dependencies' HTML docs as we can find, searchable, on our website, would be useful to have.

[sgrogan]

Yep, I think Conda is the way forward for distribution on Windows as well. Gotta work to make sure it's good on the development side of things as well. I wasn't very satisfied trying to set up a Windows dev VM and trying to Google the download links of some of the dev stuff required in the wiki... only to find it wasn't on Microsoft's website anymore. It'd be nice to have a standard Qt Creator + some non-MS compiler setup.

I think QT-Creator still uses the MS compiler under the hood. Since VS2015 I think MS offers MSBuild as a separate package that doesn't require the whole VS IDE, locally I have to "refresh" the license when prompted, maybe every 3 months? . See this post for partial progress https://forum.freecadweb.org/viewtopic. ... 51#p219051

Sure, Graphviz is nice, or perhaps something with NetworkX which I've used before on a graph theory project.

I mentioned Graphviz because FreeCAD already uses it. If an upgrade to a better package is warranted then it is a self-test of the upgrade process.

But once that's stabilized it should mostly be a matter of changing the distribution in the changelog to make builds on other platforms, and maybe some bit of patching as well.

We build FreeCAD from a single Debian Folder for all the supported Ubuntu releases. Ideally we would do the same for the dependencies we need on the PPA. On the PPA we need to be mindful of the upstream packaging. We need to decide if FreeCAD is providing an alternate or an update of the upstream(package manager) version. This goes back to a complete version of FreeCAD's dependencies(and perhaps other packages) See looo's work with Netgen6.

I think at the very least the dependency graph + as much of those dependencies' HTML docs as we can find, searchable, on our website, would be useful to have.

The links would be gravy, but the dependencies is the key. We all know "update occ, you must update nglib", I recently learned "update openssl, you must update QT and python _ssl.pyd". To build a new Libpack on Conda we must go even deeper. tcl/tk? I've built occ with this because that was how it was before on win. I think boost, occ, and python all require the same version?

So ideally the tool would help with; "I want to change package x", what do I have to do first (rebuild dependencies), and what do I have to do afterwards (rebuild things that are dependent on this).

[NormandC]

We need to decide if FreeCAD is providing an alternate or an update of the upstream(package manager) version.

Why is a decision required about this? Furthermore, what is even the difference between the two?

[sgrogan]

Why is a decision required about this? Furthermore, what is even the difference between the two?

It's about having multiple OCCT/Netgen's installed at the same time. I think FreeCAD stable can provide an upgrade and FreeCAD daily adds an alternate.