gitlab CI - stage beta, testers needed

Here's the place for discussion related to coding in FreeCAD, C++ or Python. Design, interfaces and structures.
Forum rules
Be nice to others! Respect the FreeCAD code of conduct!
User avatar
PrzemoF
Veteran
Posts: 3520
Joined: Fri Jul 25, 2014 4:52 pm
Contact:

Re: gitlab CI - stage beta, testers needed

Post by PrzemoF »

Thank you for the info! I didn't know about the free licensing Currently we don't have any CI since travis changed the rules. We never discussed the full self-hosting - for now gitlab.com + external builds, but I think FreeCAD web page is self hosted already, so maybe we could use it for the gitlab installation + external builds to keep the load at low level.

It would be great if you could help with docker. - let me know it you need any info.

The final decision belongs to the core devs - they have to be happy as it's a big move.

P.S. I have my own vested interest - I want to close my github account and the only reason I still have it is the FreeCAD source code master on github :D
User avatar
bernd
Veteran
Posts: 12849
Joined: Sun Sep 08, 2013 8:07 pm
Location: Zürich, Switzerland
Contact:

Re: gitlab - gathering info

Post by bernd »

PrzemoF wrote: Sat Jul 24, 2021 9:46 pm Successful build & test: https://gitlab.com/PrzemoF/FreeCAD/-/pi ... /342378907
I did not follow all the topics in this regard. Does it mean you are able to run the CI on your gitlab, and it passes for the commit posted?
User avatar
PrzemoF
Veteran
Posts: 3520
Joined: Fri Jul 25, 2014 4:52 pm
Contact:

Re: gitlab CI - stage beta, testers needed

Post by PrzemoF »

Big fat proud YES! :D
LeFix
Posts: 7
Joined: Sat Jul 24, 2021 9:44 am

Re: gitlab CI - stage beta, testers needed

Post by LeFix »

How do I stop the docker gitlab-runner container gracefully?

https://docs.gitlab.com/runner/best_pra ... l-shutdown
https://docs.docker.com/engine/referenc ... l/#options

Code: Select all

docker kill --signal=SIGQUIT my_container_id
doesn't work
Last edited by LeFix on Mon Jul 26, 2021 7:35 pm, edited 1 time in total.
User avatar
PrzemoF
Veteran
Posts: 3520
Joined: Fri Jul 25, 2014 4:52 pm
Contact:

Re: gitlab CI - stage beta, testers needed

Post by PrzemoF »

I started a test migration of existing pull requests from github and looks like for those that apply cleanly it's a piece of cake:
https://gitlab.com/PrzemoF/FreeCAD/-/pi ... /343198661
https://gitlab.com/PrzemoF/FreeCAD/-/pi ... /343198923
https://gitlab.com/PrzemoF/FreeCAD/-/pi ... /343199080
Not sure what about those that do not apply cleanly.

I'm not sure how multiple registered runners, but right now we have 2 and both are being used :D
Russ4262
Posts: 941
Joined: Sat Jun 30, 2018 3:22 pm
Location: Oklahoma
Contact:

Re: gitlab CI - stage beta, testers needed

Post by Russ4262 »

Afternoon Sir,
Does this new CI/CD process allow for testing Windows builds? I am not familiar with Docker or GitLab, though I have started reading through the numerous links provided earlier in this thread. Many of we FreeCADers use Windows, so I am interested in supporting Windows builds to help keep FreeCAD available on the Windows platform. The LibPacks provided by APeltauer have been instrumental. Also, the Conda builds are nice. Will this new Docker system do away with the need for the LibPacks, and follow more along the lines of the Conda builds?

Thanks,
Russell
User avatar
PrzemoF
Veteran
Posts: 3520
Joined: Fri Jul 25, 2014 4:52 pm
Contact:

Re: gitlab CI - stage beta, testers needed

Post by PrzemoF »

Hi Russell,

Check the opening post - I found 20 minutes ago Inkscape uses gitlab and yes, they have windows and macos! :D

I think I just fixed ccache - the new build took 248 ... seconds! :lol: :shock: :o
https://gitlab.com/PrzemoF/FreeCAD/-/jobs/1453779892
User avatar
bernd
Veteran
Posts: 12849
Joined: Sun Sep 08, 2013 8:07 pm
Location: Zürich, Switzerland
Contact:

Re: gitlab CI - stage beta, testers needed

Post by bernd »

PrzemoF wrote: Mon Jul 26, 2021 2:20 pm Big fat proud YES! :D
:D
User avatar
PrzemoF
Veteran
Posts: 3520
Joined: Fri Jul 25, 2014 4:52 pm
Contact:

Re: gitlab CI - stage beta, testers needed

Post by PrzemoF »

The latest version: https://gitlab.com/PrzemoF/FreeCAD/-/tree/gitlab-v3/ci

README:
It's BETA stage - limited knowledge is required or can be obtained on the FreeCAD forum.

The is the test bed to replace travis-CI that no longer is sufficient for FreeCAD CI (continuous integration)

The current (end of Jul 2021) plan is to use gitlab CI + enternal runners. External runners are nothing more than processes that build FreeCAD on external machines.

The system consists of 3 elements:
1. command and control: gitlab. Possibly can be replaced with FreeCAD own gitlab server if needed.
2. configuration files: .gitlab-ci.yml, Dockerfile and authentication token from gitlab CI settings
3. local machine executing the build

1. Currently the test aims for gitlab as the brain of the system due to convinience. All is required is a repo on gitlab (Yorik's repo is already set up). The gitlab CI can be controlled from the repo settings. If you want to help with tests please set up a fork of FreeCAD.

2. .gitlab-ci.yml tells gitlab CI how to build FreeCAD. This file contains key "image: freecadci/runner" that defines what image should be used to build FreeCAD.

3. Image freecadci/runner is defined in the Dockerfile. It contains all dependencies required to build FreeCAD, bash + text editors (vim, nano) to allow interactive debugging. The image after being locally build was pushed to dockerhub. It's stored here: https://hub.docker.com/repository/docke ... dci/runner

3. Local machine - I currently test bare metal builds (fedora). There are some configuration caveats depending on the system (selinux), so please test and report errors on the FreeCAD forum in Developiers Corner: https://forum.freecadweb.org/viewforum.php?f=10

Scripts:
register-with-gitlab - registers external runner with gitlab. To run it should should obtain the CI token first. See "Test procedure" below.
run-container - runs container (created from docker image) that executes the builds

check-all - convinience script showing all images, volumes and containers. Not required. Use to check what images, volumes and containers are on local machine.

Test procedure:
1. Install docker on your local machine
2. Fork mine repo https://gitlab.com/PrzemoF/FreeCAD or ano other FreeCAD repo and copy content od ci directory
3 Switch to branch gitlab-v3
4. In gitlab->Settings->CI/CD->General pipelines->CI/CD configuration file set location to ci/.gitlab-ci.yml
3. Copy registration token from gitlab Settings->CI/CD->Runners->Registration
5. run "register-with-gitlab registration-token"
6. run "run-container"

The above setup should create and register docker container that is ready to accept builds from gitlab.
To test is got to CI/CD->editor in gitlab and click "commit changes"

FAQ:
1. How to stop the container?
As we run container with "always restart" parameter, so to stop it it has to be changed to prevent container from beinr restarted:
docker update --restart=no <container_id>

PLEASE REPORT ALL PROBLEMS/ERRORS on the FreeCAD forum.

Well done - your read it all! :-)
Post Reply