Architecture

I am going to use the following components:

  • gentooLTO for -O3 and graphite optimizations
  • distcc to distribute the compiling jobs to a high power server
  • ccache to cache compiled files for often used libraries

Setting up the server

Updating gcc

I installed a standard gentoo installation with openrc from the default LXC packages in proxmox, where my server is. To use gentooLTO, You need to have gcc>=10, so I unmasked the unstable packet first.

/etc/portage/package.accept_keywords

sys-devel/gcc ~amd64

Then I installed the newest gcc and made it default compiler to uninstall the old one.

dbauer@appollo:~$ emerge --ask sys-devel/gcc
dbauer@appollo:~$ gcc-config
gcc-8.9.0  1
gcc-10.2.0 2
dbauer@appollo:~$ gcc-config 2
dbauer@appollo:~$ emerge --depclean

Setting up distccd

The distcc server is pretty easy to set up and get configured

dbauer@appollo:~$ emerge --ask distcc
dbauer@appollo:~$ rc-update add distccd
dbauer@appollo:~$ rc-service distccd start

The only thing left to do for distcc is now to fill the config file with some default configurations There is basically only one main line necessarry which will set all the parameters

/etc/conf.d/distccd

DISTCCD_OPTS="--port 3632 --log-level debug --log-file /var/log/distcc/distcc.log -N 15 --allow IP_ADDRESS"

The ip address you insert after --allow should be the addresses you wanna use as client which can connect to it. The number as argument -N determines the number of processes you want to start. I am on a 12 core machine, and 15 processes worked best for me, so no compiling process was blocking.

Setting up ccache

dbauer@appollo:~$ emerge --ask ccache

/var/cache/ccache/ccache.conf

# Maximum size your cache ist allowed to take up
max_size = 8.0G
#Allow others to run ebuild and share the cache
umask = 002
# preserve cache accross gcc rebuilds
compiler_check = %compiler% -v
# Rekursive number of directories
cache_dir_levels = 2

Setting up the client/desktop computer

There is a bit more work to do on the desktop computer The first thing to do is to also update to the latest gcc version. This is necassary to use gentooLTO and was needed on the server, because we need the same binutils and gcc versions on the client and server(s).

In the next step we setup distcc and make portage use it.

dbauer@appollo:~$ emerge --ask distcc

To use distcc in portage is pretty easy. We only need to activate it as a feature in our /etc/portage/make.conf Also we need to specify the local(-l) and total(-j) number of threads to use by all. A good formula should be sum of cpu-cores + 1 It is also important to specify your cpu architecture explicitly, because -march=auto wont work. This is because we need to make sure, we compile it for the correct traget architecture even when our remote server is on a different one.

/etc/portage/make.conf

GLOBALARG="-march=zenver1"
NUMTHREADS="12"
# 12 remote + 12 local + 1
MAKEOPS="-j25 -l12"
FEATURES="distcc"

One step is still missing until we can use distcc to remotely compile packages and that is to tell it where our remote servers are. We do this with a simple command

dbauer@appollo:~$ distcc-config --remote "IP_ADDR_OF_SERVER"

If you want to use ccache on your client as well, you can do the same procedure to install and setup ccache as on the server and just need to activate it as a feature in your portage/make.conf.

And that's it! It took a quite a while for me to figure it all out, but the gentoo wiki was really helpfull. I do not know if it was worth the work, but now I know, my ebuild packages are remotely compiled.