Changes at TravisCI- affecting open source

[PrzemoF]

Do you have docker or can you have docker on that machine? I'm working (with some minor successes) on setting up docker container to execute builds controlled by gitlab.

[abdullah]

My offer still stands. Realthunder has been using one of my servers for a while now and I have not gotten any complaints. As long as the finished artefacts are not hosted on my server(s) it should be fine.

Thanks. Any possibility to run a docker container?

P.S.: After several days without getting any feedback from Travis and CI operations stopped, I have little to no hope that Travis CI will be a workable solution from FreeCAD. So, this initiative has even more relevance than before.

[Cyclonit]

Sorry for the late reply, I was busy with other stuff lately.

Yes docker is not a problem. Afaik realthunder uses docker or podman to run his builds too. He might have some stuff you could reuse. Who'd need access to the machine and do you need any other resources?

[PrzemoF]

That's great! I'm polishing the edges of the docker setup (*). No access required provided you do the initial setup. I'm not sure how stable it is, but so far the image that would be running on your hardware is very stable. Resources: as much CPU as we can get. Some disk/memory obviously as well, but we can be flexible here. More is better (speed, caching, etc), but for now we have nothing, so anything that works would be great.

(*) I'm getting core dump on "FreeCAD -t 0". https://gitlab.com/PrzemoF/FreeCAD/-/jobs/1415515903

[Kunda1]

Thanks for working on this y'all!

[abdullah]

That's great! I'm polishing the edges of the docker setup (*). No access required provided you do the initial setup. I'm not sure how stable it is, but so far the image that would be running on your hardware is very stable. Resources: as much CPU as we can get. Some disk/memory obviously as well, but we can be flexible here. More is better (speed, caching, etc), but for now we have nothing, so anything that works would be great.

(*) I'm getting core dump on "FreeCAD -t 0". https://gitlab.com/PrzemoF/FreeCAD/-/jobs/1415515903

Great news!!

May the core dump be related to graphic driver stuff? Or libraries not available in the PATH when executing?

Would it be possible to get a trace pointing to the actual culprit (e.g. output of gdb / ld)?

Does FreeCADCmd -t 0 also core dump?

[PrzemoF]

Yes, the test does the same. Libraries are the problem. I can download the build (job artifacts, but I can't find where it is now, possibly expired or deleted) and local inspection suggest there is a linking problem or missing library problem:

Code: Select all

$ ldd ./FreeCAD  | grep not
	libpython3.8.so.1.0 => not found
	libCoin.so.80c => not found
	libboost_regex.so.1.71.0 => not found
	libpyside2.cpython-38-x86_64-linux-gnu.so.5.14 => not found
	libpython3.8.so.1.0 => not found
	libshiboken2.cpython-38-x86_64-linux-gnu.so.5.14 => not found
	libboost_filesystem.so.1.71.0 => not found
	libboost_program_options.so.1.71.0 => not found
	libboost_regex.so.1.71.0 => not found
	libpython3.8.so.1.0 => not found
	libboost_regex.so.1.71.0 => not found
	libpython3.8.so.1.0 => not found

Current state:
docker image: przemof/gl-v1 controls the build. This one is being run on my laptop. It's based on gitlab-runner image.
docker image image: przemof/executor executes the build. It's being run on my laptop, but gitlab takes care of it. Based on ubuntu:20.04
.gitlab-ci.yml contains bare minimum, as planned: https://gitlab.com/PrzemoF/FreeCAD/-/bl ... lab-ci.yml

[PrzemoF]

OK, enough talking, it's time to show the code: https://gitlab.com/PrzemoF/FreeCAD/-/tree/gitlab-v1/ci

Please test, report, complain, praise, suggest improvements. The testing with "FreeCAD -t 0" doesn't work, but I want to tap into the forum knowledge as I'm having problems finding the solution. It's something wiht linking or with the way binary and lids are stored in glitlab artifacts. The build artifacts (binary and the rest) can be downloaded after successful build.

[user1234]

Please ..... praise .....

Thanks for your effort! So maybe FreeCAD can drive in sight again in future.

One issue, but it have nothing to do with this directly: i can not see anything useful in the link, because gitlab forces to set cookies. So the page i see is quasi empty. That is unfortunately (for me).

Greetings
user1234

[PrzemoF]

@user1234: Thanks! OK, to do the test you need gitlab repo, so I'm not sure if you can do it without having cookies. You can always go in prviate mode or remove the cookies after the tests.

I'll post the content of the README, so at least you can have an idea what's happening:

"It's ALPHA stage - some knowledge is required or can be obtained for free on the FreeCAD forum.

The is the test bed to replace travis-CI that no longer is sufficient for FreeCAD CI (continuous integration)

The current (Jul 2021) paln is to use gitlab CI + enternal runners. External runners are nothing more that processes that build FreeCAD on external machines.
The system consists of 3 elements:
1. command and control: gitlab. Possibly can be replace with FreeCAD own gitlab server if needed.
2. configuration files: .gitlab-ci.yml, Dockerfile and authentication token from gitlab CI settings
3. local machine executing the build

1. Currently the test aims for gitlab as the brain of the system due to convinience. All is required is a repo on gitlab (Yorik's repo is already set up). The gitlan CI can be controlled form the repo settings.

2. .gitlab-ci.yml tells gitlab CI how to build FreeCAD. This file contains key "image: przemof/executor"

3. image przemof/executor is defined in Dockerfile. It contains all dependencies required to build FreeCAD. The image after being locally build was pushed to dockerhub.
Build you own image with:
docker build -t your-dockerhub-login/executor ./
or use mine

3. Local machine - I currently test bare metal builds (fedora) and virtual machine builds (ubuntu). Both seem to work. There are some configuration caveats depending on the system (selinux)

Scripts:
register-with-gitlab - registers external runner with gitlab. Contains potentially sesnsitive registration token
run-container - runs contariner (created from docker image) that executes the builds

check-all - convinience script showing all images, volumes and containers. Not required. Use to check what images, volumes and containers are on local machine.

Test provedure:
1. Install docker on your local machine
2. Fork mine repo https://gitlab.com/PrzemoF/FreeCAD
3 Switch to branch gitlab-v1
3. Obtain registration token from gitlab Settings->CI/CD->Runners->registration token and paste in register-with-gitlab script
5. run register-with-gitlab
6. run run-container

The above setup should create and register docker container that is ready to accept builds from gitlab.
To test is got to CI/CD->editor in gitlab and click "commit changes"

PLEASE REPORT ALL PROBLEMS/ERRORS on the FreeCAD forum."

Links to raw files - you can download them without accepting cookies:
Dockerfile: https://gitlab.com/PrzemoF/FreeCAD/-/ra ... Dockerfile
run-gitlab-runner: https://gitlab.com/PrzemoF/FreeCAD/-/ra ... ab-runnner
register-with-gitlab: https://gitlab.com/PrzemoF/FreeCAD/-/ra ... ith-gitlab

If you're ready to set up your own gitlab server to avoid the cookies you'll have to change the register-wit-gitlab to point to your installation or another gitlab server.