Let's talk about the test/build environment

Here's the place for discussion related to coding in FreeCAD, C++ or Python. Design, interfaces and structures.
Forum rules
Be nice to others! Respect the FreeCAD code of conduct!
User avatar
sliptonic
Veteran
Posts: 3457
Joined: Tue Oct 25, 2011 10:46 pm
Location: Columbia, Missouri
Contact:

Let's talk about the test/build environment

Post by sliptonic »

We have a PR that is failing the CI check. All the same unit tests pass locally so I don't know what to look at next. Besides asking for help on this specific issue, I thought this would be a good time to talk about the test/build environment in general.

I don't understand the current environment very well but I understand that it runs through gitlab and is executing on a machine that is owned/operated by bernd. Bernd hasn't been around for a while so I'm a little worried that we have a vulnerability with an important piece of infrastructure.

One of the primary reasons for creating the FPA was so that we could pay for key infrastructure if needed. Is it time to add capacity to our environment? What about building images for branches other than master? Toponaming?

Anyone have any thoughts or concerns about test/build?
saso wrote: pinged by pinger macro
bernd wrote: pinged by pinger macro
openBrain
Veteran
Posts: 9034
Joined: Fri Nov 09, 2018 5:38 pm
Contact:

Re: Let's talk about the test/build environment

Post by openBrain »

Actually there is some duplicate now because we have a build/test on Github CI, and the one from bernd on Gitlab.
However I think the one on Gitlab is still useful because it runs on Ubuntu 20.04 (the one on Github runs on 22.04).
I plan to propose an update of the Github CI soon so at least it also runs on "oldest supported libraries".

Then all that comes to strategical and political things.

Political : some people here had a strong refraction to have "all in Github". Such position strongly locks possibilities.

Strategical : at the end what we want with the CI/CD? Just run tests on dev branches and PR? Take care of weekly builds? Have builds for PR available as artifacts? For all platforms? Native or in a downloadable VM disk?

From there you know your needs and can talk about means. Fully external CI/CD? Based on GH with self-hosted runners? OK with only GH hosted runners?
User avatar
adrianinsaval
Veteran
Posts: 5541
Joined: Thu Apr 05, 2018 5:15 pm

Re: Let's talk about the test/build environment

Post by adrianinsaval »

github's CI is free for public repos so we could just switch to that if we're not happy with bernd's machine or if we lose it, if gitlab is preferred we also have a good amount of free minutes there I believe after https://github.com/FreeCAD/FPA/issues/13

A little gripe of mine with the current gitlab CI are the automated comments notifying about the CI run, it clutters my email, I'm interested in the actual comments and pushes in the PRs I get involved with so I usually look at those emails but many times it's just some bot notifying about stuff that we can already see in the report bellow.

On a somewhat related note, with the new "CI" on github I see a lot of annotations on unchanged parts of code when looking at the PR diffs on the web, this make it difficult to see the actual changes and I haven't found a way for hiding them for all files, I have to hide them for each file.

About building images for other branches, that's already possible and being doing with the snap packaging. It's also possible for flatpaks but we would need our own hosting infrastructure for that one. Automated windows and mac building and packaging would be nice.
User avatar
adrianinsaval
Veteran
Posts: 5541
Joined: Thu Apr 05, 2018 5:15 pm

Re: Let's talk about the test/build environment

Post by adrianinsaval »

openBrain wrote: Mon Sep 26, 2022 4:22 pm Actually there is some duplicate now because we have a build/test on Github CI, and the one from bernd on Gitlab.
is the one from github actually building? IIRC initially it didn't actually build, it just ran a bunch of checks.
I plan to propose an update of the Github CI soon so at least it also runs on "oldest supported libraries".
that's a good idea, IMO it can also be useful to do testing on "newest available" libraries so compatibility issues can be detected earl, archlinux or other rolling release like fedora rawhide or debian sid can be used for this, of course this would have lower priority and doesn't need to be a requirement for all PRs.
openBrain
Veteran
Posts: 9034
Joined: Fri Nov 09, 2018 5:38 pm
Contact:

Re: Let's talk about the test/build environment

Post by openBrain »

@adrianinsaval yes, GH CI builds and tests on 22.04. ;)
User avatar
chennes
Veteran
Posts: 3878
Joined: Fri Dec 23, 2016 3:38 pm
Location: Norman, OK, USA
Contact:

Re: Let's talk about the test/build environment

Post by chennes »

AFAICT, that CI failure for the PR is just code spell:

Code: Select all

Warning: src/Mod/Path/Path/Base/Gui/GetPoint.py:80: globalY ==> globally
Warning: src/Mod/Path/Path/Base/Gui/GetPoint.py:86: globalY ==> globally
Warning: src/Mod/Path/Path/Base/Gui/GetPoint.py:127: globalY ==> globally
Warning: src/Mod/Path/Path/Base/Gui/GetPoint.py:260: globalY ==> globally
Warning: src/Mod/Path/Path/Base/Gui/PropertyBag.py:370: nam ==> name
Warning: src/Mod/Path/Path/Base/Gui/PropertyBag.py:371: nam ==> name
Warning: src/Mod/Path/Path/Base/Gui/PropertyBag.py:372: nam ==> name
Warning: src/Mod/Path/Path/Base/Gui/PropertyBag.py:375: nam ==> name
Warning: src/Mod/Path/Path/Base/Gui/PropertyBag.py:376: nam ==> name
Warning: src/Mod/Path/Path/Base/Gui/PropertyBag.py:377: nam ==> name
Warning: src/Mod/Path/Path/Base/Gui/PropertyBag.py:380: nam ==> name
Warning: src/Mod/Path/Path/Base/Gui/PropertyBag.py:384: nam ==> name
Warning: src/Mod/Path/Path/Op/Profile.py:1095: lenE ==> lens
Warning: src/Mod/Path/Path/Op/Profile.py:1097: lenE ==> lens
Warning: src/Mod/Path/Path/Op/Profile.py:1101: lenE ==> lens
Warning: src/Mod/Path/Path/Op/Profile.py:1138: lenE ==> lens
Warning: src/Mod/Path/Path/Op/Profile.py:1164: lenE ==> lens
Warning: src/Mod/Path/Path/Op/Profile.py:1183: lenE ==> lens
Warning: src/Mod/Path/PathTests/TestPathDressupDogbone.py:45: nam ==> name
Warning: src/Mod/Path/PathTests/TestPathDressupDogbone.py:46: nam ==> name
Chris Hennes
Pioneer Library System
GitHub profile, LinkedIn profile, chrishennes.com
User avatar
adrianinsaval
Veteran
Posts: 5541
Joined: Thu Apr 05, 2018 5:15 pm

Re: Let's talk about the test/build environment

Post by adrianinsaval »

chennes wrote: Mon Sep 26, 2022 4:54 pm AFAICT, that CI failure for the PR is just code spell:
it also has:

Code: Select all

<class 'OSError'>: Error reading compression file: /usr/lib/freecad/Mod/Path/PathTests/test_filenaming.fcstd
Error: Process completed with exit code 1.
while running the tests.
mlampert
Veteran
Posts: 1772
Joined: Fri Sep 16, 2016 9:28 pm

Re: Let's talk about the test/build environment

Post by mlampert »

Anybody have an idea what I can do to make these tests pass?
I don't want to disable tests just so we can merge the PR. OTOH the PR should get in rather soon to give ppl the most time to adjust.
wmayer
Founder
Posts: 20243
Joined: Thu Feb 19, 2009 10:32 am
Contact:

Re: Let's talk about the test/build environment

Post by wmayer »

AFAICT, that CI failure for the PR is just code spell:
IMO, for source code the spell checker doesn't make really sense and produces many false-positives. I don't know if it's possible but if yes it should only check comments. And even if it has found a real typo it shouldn't happen that a build is marked as "Failed" because of that.
Anybody have an idea what I can do to make these tests pass?
When I download the file https://github.com/FreeCAD/FreeCAD/blob ... ming.fcstd I can open it with the latest master version without getting the error "Error reading compression file". But because of the changed file structures I get these expected errors:
<string>(1)<class 'ModuleNotFoundError'>: No module named 'Path.Main'; 'Path' is not a package
<string>(1)<class 'ModuleNotFoundError'>: No module named 'Path.Base'; 'Path' is not a package
<string>(1)<class 'ModuleNotFoundError'>: No module named 'Path.Main'; 'Path' is not a package
<string>(1)<class 'ModuleNotFoundError'>: No module named 'Path.Tool'; 'Path' is not a package
<string>(1)<class 'ModuleNotFoundError'>: No module named 'Path.Tool'; 'Path' is not a package
<string>(1)<class 'ModuleNotFoundError'>: No module named 'Path.Tool'; 'Path' is not a package
<string>(1)<class 'ModuleNotFoundError'>: No module named 'Path.Tool'; 'Path' is not a package
<string>(1)<class 'ModuleNotFoundError'>: No module named 'Path.Op'; 'Path' is not a package
<string>(1)<class 'ModuleNotFoundError'>: No module named 'Path.Op'; 'Path' is not a package
<string>(1)<class 'ModuleNotFoundError'>: No module named 'Path.Op'; 'Path' is not a package
<string>(1)<class 'ModuleNotFoundError'>: No module named 'Path.Main'; 'Path' is not a package
<string>(1)<class 'ModuleNotFoundError'>: No module named 'Path.Base'; 'Path' is not a package
<string>(1)<class 'ModuleNotFoundError'>: No module named 'Path.Base'; 'Path' is not a package
<string>(1)<class 'ModuleNotFoundError'>: No module named 'Path.Base'; 'Path' is not a package
<string>(1)<class 'ModuleNotFoundError'>: No module named 'Path.Tool'; 'Path' is not a package
<string>(1)<class 'ModuleNotFoundError'>: No module named 'Path.Tool'; 'Path' is not a package
<string>(1)<class 'ModuleNotFoundError'>: No module named 'Path.Tool'; 'Path' is not a package
<string>(1)<class 'ModuleNotFoundError'>: No module named 'Path.Tool'; 'Path' is not a package
<string>(1)<class 'ModuleNotFoundError'>: No module named 'Path.Op'; 'Path' is not a package
<string>(1)<class 'ModuleNotFoundError'>: No module named 'Path.Op'; 'Path' is not a package
<string>(1)<class 'ModuleNotFoundError'>: No module named 'Path.Op'; 'Path' is not a package
In the stack trace there appears this error: "invalid document structure" Any idea where this message comes from? Is it part of your PR?

Does a local run work for you? If yes, do you use FreeCAD or FreeCADCmd?
User avatar
wandererfan
Veteran
Posts: 6268
Joined: Tue Nov 06, 2012 5:42 pm
Contact:

Re: Let's talk about the test/build environment

Post by wandererfan »

sliptonic wrote: Mon Sep 26, 2022 3:45 pm Anyone have any thoughts or concerns about test/build?
Builds on all platforms, please. Finding out that a PR doesn't build in Windows after merge is frustrating to all concerned.
Post Reply