Compiling with distcc (distribute + speed-up compiling FreeCAD across several machines)

[tripstar76]

Hi, and hope you're all keeping well.

Right, I'm at my wit's end trying to compile using distcc.
Does anyone have a guaranteed to work, set up and walkthrough of installing distcc, and whatever deps, setting up the master and slave nodes, and then the compiling of FreeCAD.

I've tried numerous guides and tutorials, but they are either old, or confusing!

I can compile using the normal compile, but time is precious haha.

The computers I'm using are 2 Linux mint 20, and the latest Debian.
I'm coming the latest git of FreeCAD.

Hopefully someone will be able to help, as I've had a wasted day, and nearly a night of trying.

Thanks, John

[Kunda1]

Sorry no one responded to this. I don't think anyone uses distcc even though it makes so much blooming sense to! Are you still interested in testing this?

[kkremitzki]

I've tried to get it working a bit..

[Kunda1]

I've tried to get it working a bit..

What have you done so far and where are the snags?

[Kunda1]

So what we would need is instructions for users who have 2 or more machines that are interested in builidng FreeCAD on. Distcc (INSTALL) requires that, in short:

• For each machine, download distcc unpack, and do
  ./configure && make && sudo make install
• On each of the servers, run distccd --daemon, with --allow options to restrict access.
• Put the names of the servers in your environment:
  export DISTCC_POTENTIAL_HOSTS='localhost red green blue'
• Build!
• Wrap your build command in the "pump" script and use "distcc" as your C compiler:
  cd ~/work/myproject; pump make -j8 CC=distcc

Theoretically, once distcc is installed on the the various machines, we'd just then need to plug in the instructions from Compile on Linux for the build to work, right ?

[etrombly]

Just noticed this, I use distcc with docker. On the remote machines I use the following dockerfile:

Code: Select all

FROM registry.gitlab.com/daviddaish/freecad_docker_env:latest

RUN apt-get update && \
  apt-get install -y \
    distcc distcc-pump procps g++-6 gcc-6 && \
  rm -rf /var/lib/apt/lists/* && \
  chmod 777 /tmp

# This is the operations port
EXPOSE 3632
# This is the statistics port
EXPOSE 3633

USER distccd

ENTRYPOINT /usr/bin/distccd --no-detach --daemon --stats --log-level info --log-stderr $OPTIONS

I use docker-compose, so add the following entry to my docker-compose.yaml

Code: Select all

  distcc:
    build: /storage/docker/distcc
    container_name: distcc
    environment:
      OPTIONS: "--allow 192.168.0.0/24"
    ports:
      - 3632:3632
      - 3633:3633

Then on the machine I run the compilation from I use this dockerfile

Code: Select all

FROM registry.gitlab.com/daviddaish/freecad_docker_env:latest

RUN chmod 777 /tmp && \
  apt-get update && \
  apt-get install -y \
  distcc distcc-pump procps ccache && \
  rm -rf /var/lib/apt/lists/*

COPY hosts /etc/distcc/hosts
COPY build_debug.sh /root/build_debug.sh
COPY FreeCAD /root/.FreeCAD

the hosts file looks like

Code: Select all

# As described in the distcc manpage, this file can be used for a global
# list of available distcc hosts.
#
# The list from this file will only be used, if neither the
# environment variable DISTCC_HOSTS, nor the file $HOME/.distcc/hosts
# contains a valid list of hosts.
#
# Add a list of hostnames in one line, seperated by spaces, here.
localhost/6 192.168.0.99/10,lzo 192.168.0.98/10,lzo

the number after the slash is the number of jobs to launch on that host. The build_debug file is only really needed if you want to make debug builds. If you want a regular build you can just modify the build_script.sh in the container so it has a line like

Code: Select all

    -D CMAKE_C_COMPILER_LAUNCHER=distcc -D CMAKE_CXX_COMPILER_LAUNCHER=distcc \

[Kunda1]

Just noticed this, I use distcc with docker. On the remote machines I use the following dockerfile:

Awesome! Questions:
How many machines are you using to compile FC on?
How much does this speed up compilation time?

[Kunda1]

Interesting, on the Arch freecad-git AUR package discussion apparently there is a bug on FreeCAD's side of things
https://aur.archlinux.org/packages/free ... ent-782379

Doesn't build with both ccache and distcc enabled. The CMakeLists.txt has some custom logic to use ccache in front of every compilation command if it's found, which combined with makepkg's logic to set up the PATH for ccache results in invocations like /usr/bin/ccache /usr/lib/ccache/bin/c++. With distcc also enabled, both the "outer" and "inner" ccache programs trigger distcc which causes it to bail out with "CRITICAL! distcc seems to have invoked itself recursively!".

Code: Select all

diff --git CMakeLists.txt CMakeLists.txt
index f4dabf7657..ded1023532 100644
--- CMakeLists.txt
+++ CMakeLists.txt
@@ -8,11 +8,6 @@ if (POLICY CMP0072)
     cmake_policy(SET CMP0072 OLD)
 endif(POLICY CMP0072)

-find_program(CCACHE_PROGRAM ccache)  #This check should occur before project()
-if(CCACHE_PROGRAM)
-    set_property(GLOBAL PROPERTY RULE_LAUNCH_COMPILE "${CCACHE_PROGRAM}")
-endif()
-
 project(FreeCAD)

 set(PACKAGE_VERSION_NAME "Vulcan")
 

[etrombly]

I'm running it on 3 machines, I can compile in about 30 minutes. Haven't built in a little while, I've been using the appimage. Let me verify everything is still working correctly.

[Kunda1]

I'm running it on 3 machines, I can compile in about 30 minutes. Haven't built in a little while, I've been using the appimage. Let me verify everything is still working correctly.

30 minutes for 3 machines, that is higher than I expected.

edit: it does make sense though, since it take roughly about 1.5h to build on a single machine.