Skip to content

How to set up a GitLab runner

GitLab runners are software agents which run GitLab continuous integration jobs on private servers. They are usually used to build, test and deploy software products.

Wherever possible, Beautiful Canoe projects should use the Docker infrastructure that is available on GitLab to execute CI/CD jobs, and avoid installing runners on our own servers. However, where a project needs to serve a web application, it will need a self-hosted GitLab runner to manage its deployment.

Installing GitLab runner

GitLab runner requires a non-standard Ubuntu repository as an apt source, before it can be installed:

curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh | sudo bash
sudo apt-get install gitlab-runner

Installing the gitlab-runner package will also create a new user called gitlab-runner, however, this user may not have a home directory. Adding a $HOME directory will mean that gitlab-runner can be run as a non-privileged user:

mkhomedir_helper gitlab-runner

Override the default systemd configuration

gitlab-runner can be started via systemd with sudo service gitlab-runner start or restart. This loads a default configuration, which is stored in /etc. However, we only want gitlab-runner to access files that it owns, outside the system directories.

To make this work, run:

sudo systemctl edit gitlab-runner

which will open up your default editor. This text will be stored in the file /etc/systemd/system/gitlab-runner.service.d/override.conf. This ensures that our changes will not be overwritten the next time apt-get upgrades the gitlab-runner package.

Add these lines to the new file:

[Service]

ExecStart=
ExecStart=/usr/bin/gitlab-runner "run" "--working-directory" "/home/gitlab-runner" "--config" "/home/gitlab-runner/.gitlab-runner/config.toml" "--service" "gitlab-runner" "--user" "gitlab-runner"

Notice that we need the section heading [Service] and also to clear ExecStart before we add the new configuration.

In future, when we add new runners on this server, we can do that by suing to the gitlab-runner user. Configuration for new runners will be stored in $HOME/.gitlab-runner/config.toml, but we can still manage the service with systemd.

Running pipelines on Docker

If you are running pipelines on Docker, there are two more steps to take. Firstly, locate the config.toml file for the new runner. It is likely to be either in /etc/gitlab-runner/config.toml if you are running gitlab-runner as root, or /home/gitlab-runner/.gitlab-runner/config.toml if you are running the service as gitlab-runner.

Edit the file, and add:

pull_policy = "if-not-present"

to the runners.docker configuration. This ensures that Docker will use an image held locally on the server, if there is one.

Secondly, add a new file in /etc/cron.weekly/ called docker-prune, with this contents:

#!/bin/sh

set -e

docker system prune -a --volumes --force
docker network prune --force

and make it executable:

sudo chmod +x /etc/cron.weekly/docker-prune

This ensures that the server will not run out of disk space, even though gitlab-runner does not prune its own images.

Register a new runner

To register a new runner with GitLab you will need a token. This can be found on the CI/CD configuration page of your GitLab repository: https://gitlab.com/beautifulcanoe/projects/PROJECT/settings/ci_cd#js-runners-settings.

Registering the runner is an interactive process, it should look something like this:

$ sudo gitlab-ci-multi-runner register

Runtime platform                                    arch=amd64 os=linux pid=29567 revision=de08a4bb version=11.9.1
Running in system-mode.

Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/):
https://gitlab.com/

Please enter the gitlab-ci token for this runner:
TOKEN_FROM_CI_CD_PAGE

Please enter the gitlab-ci description for this runner:
PROJECT-ENVIRONMENT

Please enter the gitlab-ci tags for this runner (comma separated):
demo,staging

Registering runner... succeeded                     runner=EygBBkzs

Please enter the executor: ssh, virtualbox, docker+machine, docker-ssh+machine, docker, shell, kubernetes, docker-ssh, parallels:
shell

Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded!

image

Check back on your CI/CD configuration page, and you should now see the new runner, with whichever name and tags you have chosen to give it. If you click on the runner ID, you will see a page with some extra options that you might want to consider using, in particular:

  • This runner will only run on pipelines triggered on protected branches
  • Indicates whether this runner can pick jobs without tags

image

Important

If your new runner will deploy a public URL, please add it to the company dashboard on uptimerobot for live monitoring, and ensure that alerts go to alert@beautifulcanoe.com and to the relevant -project Slack channel (or bc-online if your project is internal). For the Slack alert, you will need to create a new webhook in our Slack app. If you don't have access to our uptimerobot account, or our Slack app, please ask the CTO. If your site has an SSL certificate, please also raise an issue and an MR to add it to our SSL validation script in the cron repository. Your SSL certificate should also be renewed automatically