GSoC '18: CM Project Thread

Contributions from the participants, questions and answers to their projects.
Discussions of proposals for upcoming events.
Forum rules
Be nice to others! Respect the FreeCAD code of conduct!
User avatar
kkremitzki
Veteran
Posts: 2511
Joined: Thu Mar 03, 2016 9:52 pm
Location: Illinois

Re: GSoC '18: CM Project Thread

Post by kkremitzki »

Hi folks, just wanted to post a quick update about my first few days.

Monday: I had previously seen a blog series about analyzing Debian packages using a graph database which seemed intriguing and possibly useful here. This work uses UDD, the Ultimate Debian Database, which includes tons of meta-information about packages, bugs, and so forth, stored in a PostgreSQL database. There's a 1.1 GB dump available, generated every 2 days. It was fairly straightforward to set up a Postgres 10 server on my workstation to import this data although it comes from a version 9 source. I spent some time poking around at the data with pgAdmin 3 and pgcli.

I also found a package called debtree which, along with graphviz, can be used to generate a complete dependency graph, but to be honest the output is so complex it's practically useless for imparting any understanding. I need to look further into these and probably do some pruning to have something useful.

Tuesday & Wednesday: Mostly spent in early phases of PySide 2 packaging. This is actually going to be 3 source packages, pyside2, shiboken2, and pyside2-tools, with probably about 40 binary packages total. I think working on it should be one of my main focuses in the first coding period.

Additionally, I missed testing gmsh against my OCCT 7.2 package, so I was trying to get that working but was stuck on some build failures. My work-in-progress package for Netgen, based on 6.2.1802, was due for an update since 6.2.1804 is now out, but it's also failing to build currently.

Today: It's no doubt that the Debian packaging ecosystem is complex. Up til now, I had been using the wrapper-of-wrappers tool pbuilder. However, I had heard some about sbuild as an alternative and wanted to give it a shot. So far, it seems like a big step up! It's very straightforward to have a build environment which has all the bells and whistles you'd want: apt package caching, running the whole source & build directory on a RAM tmpfs, ccache integration, and great integration with other Debian packaging tools like lintian, piuparts, and autopkgtest. There's some good documentation over at https://wiki.ubuntu.com/SimpleSbuild.

Thus far today, I've been experimenting with sbuild and continuing my work from Tuesday & Wednesday.

----

As an aside... my OCCT 7.2 package has been in the Debian NEW queue for about a month and a half. In some extreme cases there are much older packages in there, as old as 7 months. Considering how much of what I want to accomplish this summer depends on that package being available, I wonder if before too long I shouldn't try to contact someone about getting it looked at...
Like my FreeCAD work? I'd appreciate any level of support via Patreon, Liberapay, or PayPal! Read more about what I do at my blog.
User avatar
yorik
Founder
Posts: 13640
Joined: Tue Feb 17, 2009 9:16 pm
Location: Brussels
Contact:

Re: GSoC '18: CM Project Thread

Post by yorik »

Looks good so far, don't forget to work a bit on FreeCAD too :D Joking apart, packaging is of course an important part, but as you said yourself there is still a big part of "debian bureaucracy" before a package is accepted, over which you'll have no control. So I think you should keep this as a side thing and not throw all your time in it, because it could end up with nothing at all.

Sean is insisting that you have a clear plan with goals... Could you make that somehow? This thread is not really a good/easy way to keep track of your progresses, for someone who's not reading it all the time, and we'll need a good way for other people like him to assess the progresses.
User avatar
kkremitzki
Veteran
Posts: 2511
Joined: Thu Mar 03, 2016 9:52 pm
Location: Illinois

Re: GSoC '18: CM Project Thread

Post by kkremitzki »

yorik wrote: Fri May 18, 2018 7:56 pm Looks good so far, don't forget to work a bit on FreeCAD too :D Joking apart, packaging is of course an important part, but as you said yourself there is still a big part of "debian bureaucracy" before a package is accepted, over which you'll have no control. So I think you should keep this as a side thing and not throw all your time in it, because it could end up with nothing at all.

Sean is insisting that you have a clear plan with goals... Could you make that somehow? This thread is not really a good/easy way to keep track of your progresses, for someone who's not reading it all the time, and we'll need a good way for other people like him to assess the progresses.
I updated my profile page with a plan, can you take a look and let me know if you have any feedback: http://brlcad.org/wiki/User:Kkremitzki

By the way, OCCT 7.2 got accepted into Debian experimental! Now that it has spent its time in the NEW queue, subsequent uploads & improvements can be automatic. Additionally, that means I can get moving on all the stuff depending on it.

The only other packages that will have to suffer the NEW queue are the PySide 2 packages and SMESH, so I think I should front-load work on those. There are a LOT of people anticipating PySide 2 though so I think once I get that in, the wait shouldn't be long.
Like my FreeCAD work? I'd appreciate any level of support via Patreon, Liberapay, or PayPal! Read more about what I do at my blog.
abdullah
Veteran
Posts: 4935
Joined: Sun May 04, 2014 3:16 pm
Contact:

Re: GSoC '18: CM Project Thread

Post by abdullah »

kkremitzki wrote: Mon May 21, 2018 2:34 pm By the way, OCCT 7.2 got accepted into Debian experimental! Now that it has spent its time in the NEW queue, subsequent uploads & improvements can be automatic. Additionally, that means I can get moving on all the stuff depending on it.

The only other packages that will have to suffer the NEW queue are the PySide 2 packages and SMESH, so I think I should front-load work on those. There are a LOT of people anticipating PySide 2 though so I think once I get that in, the wait shouldn't be long.
Congrats!! It is quite a milestone. The work pays off eventually, but it takes a while....
User avatar
yorik
Founder
Posts: 13640
Joined: Tue Feb 17, 2009 9:16 pm
Location: Brussels
Contact:

Re: GSoC '18: CM Project Thread

Post by yorik »

Thanks for updating your plan! Looks ok to me.
Excellent news with opencascade! Thanks for the hard work! Nice view now :D
Screenshot from 2018-05-25 18-24-40.png
Screenshot from 2018-05-25 18-24-40.png (102.24 KiB) Viewed 1182 times
I'm also talking about you to some debian science people so when you arrive with pyside2 some of them will hopefully be watching already ;)
User avatar
NormandC
Veteran
Posts: 18587
Joined: Sat Feb 06, 2010 9:52 pm
Location: Québec, Canada

Re: GSoC '18: CM Project Thread

Post by NormandC »

Great news!

I hope you can figure out the whole pyside2/netgen/smesh thing by the end of summer. ;)
User avatar
kkremitzki
Veteran
Posts: 2511
Joined: Thu Mar 03, 2016 9:52 pm
Location: Illinois

Re: GSoC '18: CM Project Thread

Post by kkremitzki »

Late update post!

Last week I mainly focused on PySide 2. As I had mentioned, PySide 2 is really three source projects, pyside2, shiboken2, and pyside2-tools. In Debian, this is reflected in the source packages for pyside 1--there are three separate source packages for the three components. However, upstream ships them together in PySide 2 using a combination of git modules (for pyside2-tools) and by simply including a shiboken2 folder inside the sources folder of the repository.

On top of that, they use a mixture of Python's setuptools and cmake to build everything, which really complicates things from the Debian perspective. However, in my testing, I found it was basically possible to take sources/pyside2, sources/shiboken2, and sources/pyside2-tools, move them to their own folders, bring in copies of build_scripts, and build them by themselves just using cmake while just chucking the rest of the repo. This lets me build packages for all three sources, and things work as expected when using PySide 2 built this way. However, because it requires a good bit of munging of the source, it might generate some pushback.

Luckily, I don't think it's something I will need to worry about immediately. I found out that Raphaël Hertzog of Freexian has been contracted to work on packaging PySide 2, so I suspect that task will largely be off my plate. I got in touch with him to coordinate work and he referred me to Debian bug #877871: ITP: pyside2 -- Qt for Python where you can read more about the ongoing work to package PySide 2. The key takeaway is that Raphaël and Sophie, who I presume is his wife, will be working on it together and they intend to "do it the hard way" with one monolithic pyside2 source repository.

Other big news: the current maintainers of FreeCAD on Debian decided to remove their names from the package, which would've been the first step towards abandoning it, but about three other people piped up in the mailing list expressing their interest to help, besides myself. (I waited to reply because I wanted to let others who were interested come out and say so.)

Currently, FreeCAD 0.16 is failing to build from source (FTBFS), so before proceeding that needs to be addressed. The cause is the gdal-2.3.0 transition, see https://tracker.debian.org/pkg/gdal. It's important to maintain the ability of v0.16 to be built so that security backports, etc., can be provided if necessary.

Also, over the weekend, some work was done by Tobias Frost on cleaning up OCCT 7.2.0's symbols files, which along with some other things were preventing it from building on different Debian platforms. Now that the package has had a little bit of cleanup, I think this week is a good time to get the first FreeCAD 0.17 built against it pushed into Debian experimental. Then, later on, as improvements become available (e.g. netgen support again) those improvements can be tacked on to the package.

P.S. As looo mentioned here I forgot to include pivy and soqt-qt5 on the list of todo items. I'm going to look more into what will be required to package these this week.
Like my FreeCAD work? I'd appreciate any level of support via Patreon, Liberapay, or PayPal! Read more about what I do at my blog.
User avatar
Kunda1
Veteran
Posts: 13434
Joined: Thu Jan 05, 2017 9:03 pm

Re: GSoC '18: CM Project Thread

Post by Kunda1 »

Image
Alone you go faster. Together we go farther
Please mark thread [Solved]
Want to contribute back to FC? Checkout:
'good first issues' | Open TODOs and FIXMEs | How to Help FreeCAD | How to report Bugs
User avatar
yorik
Founder
Posts: 13640
Joined: Tue Feb 17, 2009 9:16 pm
Location: Brussels
Contact:

Re: GSoC '18: CM Project Thread

Post by yorik »

Excellent! pyside2 in debian will be a huge step forward...
For pivy make sure to use loooo's (never know how many o :D ) repo, he made a couple of changes for python3 and has become the de facto sole maintainer of pivy...
User avatar
kkremitzki
Veteran
Posts: 2511
Joined: Thu Mar 03, 2016 9:52 pm
Location: Illinois

Re: GSoC '18: CM Project Thread

Post by kkremitzki »

Another update that I will elaborate on a bit later today:

- a nicer gmsh with occt 7.2 will be sent out for consideration today.
- as occt 7.3 is out, I spent some time preparing the packaging for it. I think I ran into a bug somewhere in the git-buildpackage toolchain; it was choking on a file, the succinctly named

Code: Select all

src/StepVisual/StepVisual_AnnotationCurveOccurrenceAndAnnotationOccurrenceAndGeomReprItemAndReprItemAndStyledItem.hxx
but I got past it and should have that package by this weekend.
- I am also working on the FreeCAD 0.17 package with OCCT 7.2 support, but the builds are failing for the same reason my netgen 6.2.1804 ones are: a tough-to-troubleshoot error message involving ld not finding -lpthreads. There are reports of the same/similar error message in the forum but in wildly different environments (e.g. Mac.) The troubleshooting persists on this one... perhaps something is broken in Debian unstable? Regular compiling works...
- I looked into and got started with the packaging for pivy and soqt. I was less familiar with these parts of FreeCAD as evidenced by the fact I forgot to include them in the first place. Since we already have a github.com/freecad/pivy we should get an official one for soqt as well. Both then will need a tagged release to help make our fork more official. We are the only thing reverse-depending on soqt and pivy in Debian that I can find.
looo wrote:ping
Lorenz would it be possible for you to do the tagging and move your soqt fork into github.com/freecad?

P.S. Finally, the Texas Linux Fest presentation is coming up this Saturday!
Like my FreeCAD work? I'd appreciate any level of support via Patreon, Liberapay, or PayPal! Read more about what I do at my blog.
Post Reply