Skip to content

How to set up marge-bot

marge-bot is an automated service that merges GitLab merge requests. When a merge-request is ready, just assign it to its user, and Marge will do all any necessary rebase-wait-retry for you. If anything goes wrong (merge conflicts, tests that fail, etc.) Marge will leave a comment on the merge-request.

Install docker

On the server that will run marge-bot, install Docker:

sudo apt-get update
sudo apt-get install docker.io

Create a non-privileged user

Create a new user, with a home directory, and add the new user to the docker group:

sudo adduser --disabled-password marge-bot
sudo usermod -a -G docker marge-bot

Create credentials

To identify the instance of marge-bot on GitLab you will need to create some private credentials.

SSH key

On the server that will be running the bot, create a new SSH key (replace the email address with a valid address):

sudo su marge-bot
cd
ssh-keygen -t ed25519 -C marge-bot@invalid -f marge-bot-ssh-key -P ''

You will need both the public and private keys in later steps.

GitLab personal access token

Go to the personal access tokens page in the settings of your GitLab user account (probably @bc-bot). Create a new token with api level access, and store the token somewhere safe (do not close the browser window until you are finished).

This token gives @bc-bot access to the GitLab API for all the groups and projects to which the bot user belongs, so that it can see which merge requests are assigned to @bc-bot.

Upload the public SSH key to GitLab

Go to the ssh keys page in the settings of your GitLab user account (probably @bc-bot). Add the new public key to the list of keys for the bot, and give it a name that makes it clear that the key will be used by marge-bot, and which server will be used.

Create a marge-bot configuration

Create a directory to hold a configuration file:

sudo su marge-bot
cd
mkdir config

Place the new personal access token in a file called /home/marge-bot/config/token. Place the private SSH key in a file called /home/marge-bot/config/marge-bot-ssh-key.

Create a file called /home/marge-bot/config/bc-bot-config.yml and add in a basic configuration:

# File /home/marge-bot/config/bc-bot-config.yml
debug: false
add-part-of: true
# add-reviewers requires a self-hosting GitLab instance.
add-reviewers: false
add-tested: true
# chose one way of specifying the Auth token
# auth-token:
auth-token-file: /config/token
branch-regexp: .*
ci-timeout: 60min
embargo: Friday 6pm - Monday 8am
batch: false
git-timeout: 600s
gitlab-url: "https://gitlab.com"
# impersonate-approvers requires a self-hosting GitLab instance.
impersonate-approvers: false
project-regexp: .*
# chose one way of specifying the SSH key
ssh-key-file: /config/marge-bot-ssh-key
# ssh-key:

Note that we use files to store the PAT and SSH key, which are mounted in the Docker volume. This is to avoid leaking secrets to other users on the server who can run docker ps.

The front page of the marge-bot GitHub page has more documentation on configuring the bot.

Test marge-bot

Check that marge-bot runs correctly in docker:

docker run -v /home/marge-bot/config/:/config \
  -v /etc/localtime:/etc/localtime:ro \
  smarkets/marge-bot \
  --config-file=/config/bc-bot-config.yml

Note that -v /etc/localtime:/etc/localtime:ro mounts the localtime file as read-only, which ensures that marge-bot always runs in a Docker container with the same timezone as the host server.

Don't worry if marge-bot takes a while to start, but once it does you should see a stream of output like this:

Mar 09 17:53:52 bc-web01d-vm docker[24526]: 2019-03-09 17:53:52,542 INFO Nothing to merge at this point...
Mar 09 17:53:53 bc-web01d-vm docker[24526]: 2019-03-09 17:53:53,543 INFO Fetching merge requests assigned to me in beautifulcanoe/projects/actss...
Mar 09 17:53:54 bc-web01d-vm docker[24526]: 2019-03-09 17:53:54,433 INFO Nothing to merge at this point...

Use Ctrl+c to stop the bot running before moving to the next stage.

Create a systemd service

Once marge-bot works, you will want to run it as a background daemon that starts automatically when the server reboots.

Create a new file called /etc/systemd/system/marge-bot.service to hold the systemd configuration. Your new service will need to run as the marge-bot user and group, after docker.service has started.

You will also need to include an [Install] section in the configuration, in order to enable the service to start on boot.

Since marge-bot is running in Docker, it is helpful to give the container a name (i.e. --name=marge) so that it is easier to stop or rm the container via systemd.

For example:

[Unit]
Description=marge-bot merge robot
ConditionPathExists=/home/marge-bot/config
After=docker.service
Requires=docker.socket

[Service]
User=marge-bot
Group=marge-bot
ExecStartPre=/usr/bin/docker pull smarkets/marge-bot
ExecStart=/usr/bin/docker run --name=marge -v /home/marge-bot/config/:/config -v /etc/localtime:/etc/localtime:ro smarkets/marge-bot --config-file=/config/bc-bot-config.yml
ExecStop=/usr/bin/docker stop marge
ExecStopPost=/usr/bin/docker rm -f marge
KillMode=process
Restart=on-failure
RuntimeDirectory=/home/marge-bot/
RuntimeDirectoryMode=0755
TimeoutSec=infinity

[Install]
WantedBy=multi-user.target
Alias=marge-bot.service

Enable and start the bot:

sudo systemctl enable marge-bot.service
sudo systemctl start marge-bot.service

and check that the bot is running:

$ sudo systemctl status marge-bot
● marge-bot.service - marge-bot merge robot
   Loaded: loaded (/etc/systemd/system/marge-bot.service; enabled; vendor preset: enabled)
   Active: active (running) since Wed 2019-05-29 11:33:43 BST; 2s ago
  Process: 11668 ExecStopPost=/usr/bin/docker rm -f marge (code=exited, status=0/SUCCESS)
  Process: 11340 ExecStop=/usr/bin/docker stop marge (code=exited, status=0/SUCCESS)
  Process: 11676 ExecStartPre=/usr/bin/docker pull smarkets/marge-bot (code=exited, status=0/SUCCESS)
 Main PID: 11742 (docker)
    Tasks: 10 (limit: 4662)
   CGroup: /system.slice/marge-bot.service
           └─11742 /usr/bin/docker run --name=marge -v /home/marge-bot/config/:/config -v /etc/localtime:/etc/localtime:ro smarkets/marge-bot --config-file=/config/bc-bot-config.yml

May 29 11:33:41 bc-web01d-vm systemd[1]: Starting marge-bot merge robot...
May 29 11:33:41 bc-web01d-vm docker[11676]: Using default tag: latest
May 29 11:33:43 bc-web01d-vm docker[11676]: latest: Pulling from smarkets/marge-bot
May 29 11:33:43 bc-web01d-vm docker[11676]: Digest: sha256: ...
May 29 11:33:43 bc-web01d-vm docker[11676]: Status: Image is up to date for smarkets/marge-bot:latest
May 29 11:33:43 bc-web01d-vm systemd[1]: Started marge-bot merge robot.

Troubleshooting

Keeping marge-bot up to date

Occasionally marge-bot can get stuck, and will need to be restarted. Also, the marge-bot code is under development, and we (currently) run the latest version of the Docker container. Therefore, it is sensible to restart the marge-bot service relatively regularly. This should be done via systemd to ensure that the Docker container is correctly removed.

Currently, we have a cron job in /etc/cron.daily like this:

#!/bin/sh

service marge-bot restart

There can be only one marge-bot

There should be only one instance of marge-bot running at any one time. However, docker containers are not automatically cleaned up. If you have only started marge-bot from the systemd configuration above, you should only have one marge-bot running, and its container should be called marge. You can check by running sudo docker -ps -a, and you should see something like:

$ sudo docker ps --all
CONTAINER ID        IMAGE                COMMAND                  CREATED             STATUS              PORTS               NAMES
f8a907df4d16        smarkets/marge-bot   "/bin/marge.app --co..."   45 minutes ago      Up 45 minutes                           marge

If you only want to see the container names, try sudo docker ps --all --format "table {{.Names}}".

If marge-bot has restarted itself, or you have created a number of containers whilst testing the bot, make sure to remove them with sudo docker rm -f NAME.

To automatically detect extra instances of marge-bot running, add a script like the one below to /etc/cron.hourly/:

#!/bin/bash

names=$(docker ps -all --filter ancestor=smarkets/marge-bot --format "table {{.Names}}" | tail -n +2 | grep -v marge)

for name in $names
do
    echo "Killing marge-bot Docker container $name"
    docker rm -f $name
done