Skip to content

How to set up marge-bot

marge-bot is an automated service that merges GitLab merge requests. When a merge-request is ready, just assign it to its user, and marge will do all any necessary rebase-wait-retry for you. If anything goes wrong (merge conflicts, tests that fail, etc.) marge will leave a comment on the merge-request.

Install docker

On the server that will run marge-bot, install Docker:

sudo apt-get update
sudo apt-get install docker.io

Create a non-privileged user

Create a new user, with a home directory, and add the new user to the docker group:

sudo adduser marge-bot
sudo usermod -a -G docker marge-bot

Create credentials

To identify the instance of marge-bot on GitLab you will need to create some private credentials.

SSH key

On the server that will be running the bot, create a new SSH key (replace the email address with a valid address):

sudo su marge-bot
cd
ssh-keygen -t ed25519 -C marge-bot@invalid -f marge-bot-ssh-key -P ''

You will need the private key in a later step.

GitLab personal access token

Go to the personal access tokens page in the settings of your GitLab user account (probably @bc-bot). Create a new token with api level access, and store the token somewhere safe (do not close the browser window until you are finished).

Create a marge-bot configuration

Create a directory to hold a configuration file:

sudo su marge-bot
cd
mkdir config

Create a file called /home/marge-bot/config/bc-bot-config.yml and add in a basic configuration. The example here is a good start, and you will need to include the personal access token and (private) SSH key from the previous steps.

# File /home/marge-bot/config/bc-bot-config.yml

add-part-of: true
# add-reviewers requires a self-hosting GitLab instance.
add-reviewers: false
add-tested: true
auth-token: itsasecret
branch-regexp: .*
ci-timeout: 15min
embargo: Friday 5pm - Monday 9am
batch: false
git-timeout: 120s
gitlab-url: "https://gitlab.com"
# impersonate-approvers requires a self-hosting GitLab instance.
impersonate-approvers: false
project-regexp: beautifulcanoe/.*
ssh-key: |
  -----BEGIN OPENSSH PRIVATE KEY-----
  ...
  -----END OPENSSH PRIVATE KEY-----

The front page of the marge-bot GitHub page has more documentation on configuring the bot.

Test marge-bot

Check that marge-bot runs correctly in docker:

docker run -v /home/marge-bot/config/:/config \
  -v /etc/localtime:/etc/localtime:ro \
  smarkets/marge-bot \
  --config-file=/config/bc-bot-config.yml

Note that -v /etc/localtime:/etc/localtime:ro mounts the localtime file as read-only, which ensures that marge-bot always runs in a Docker container with the same timezone as the host server.

Don't worry if marge-bot takes a while to start, but once it does you should see a stream of output like this:

Mar 09 17:53:52 bc-web01d-vm docker[24526]: 2019-03-09 17:53:52,542 INFO Nothing to merge at this point...
Mar 09 17:53:53 bc-web01d-vm docker[24526]: 2019-03-09 17:53:53,543 INFO Fetching merge requests assigned to me in beautifulcanoe/projects/actss...
Mar 09 17:53:54 bc-web01d-vm docker[24526]: 2019-03-09 17:53:54,433 INFO Nothing to merge at this point...

Use Ctrl+c to stop the bot running before moving to the next stage.

Create a systemd service

Once marge-bot works, you will want to run it as a background daemon that starts automatically when the server reboots.

Create a new file called /etc/systemd/system/marge-bot.service to hold the systemd configuration. Your new service will need to run as the marge-bot user and group, after docker.service has started.

You will also need to include an [Install] section in the configuration, in order to enable the service to start on boot.

Since marge-bot is running in Docker, it is helpful to give the container a name (i.e. --name=marge) so that it is easier to stop or rm the container via systemd.

For example:

[Unit]
Description=marge-bot merge robot
ConditionPathExists=/home/marge-bot/config
After=docker.service
Requires=docker.socket

[Service]
User=marge-bot
Group=marge-bot
ExecStartPre=/usr/bin/docker pull smarkets/marge-bot
ExecStart=/usr/bin/docker run --name=marge -v /home/marge-bot/config/:/config g -v /etc/localtime:/etc/localtime:ro smarkets/marge-bot --config-file=/config/bc-bot-config.yml
ExecStop=/usr/bin/docker stop marge
ExecStopPost=/usr/bin/docker rm -f marge
KillMode=process
Restart=on-failure
RuntimeDirectory=/home/marge-bot/
RuntimeDirectoryMode=0755
TimeoutSec=infinity

[Install]
WantedBy=multi-user.target
Alias=marge-bot.service

Enable and start the bot:

sudo systemctl enable marge-bot.service
sudo systemctl start marge-bot.service

and check that the bot is running:

$ sudo systemctl status marge-bot
● marge-bot.service - marge-bot merge robot
   Loaded: loaded (/etc/systemd/system/marge-bot.service; enabled; vendor preset: enabled)
   Active: active (running) since Wed 2019-05-29 11:33:43 BST; 2s ago
  Process: 11668 ExecStopPost=/usr/bin/docker rm -f marge (code=exited, status=0/SUCCESS)
  Process: 11340 ExecStop=/usr/bin/docker stop marge (code=exited, status=0/SUCCESS)
  Process: 11676 ExecStartPre=/usr/bin/docker pull smarkets/marge-bot (code=exited, status=0/SUCCESS)
 Main PID: 11742 (docker)
    Tasks: 10 (limit: 4662)
   CGroup: /system.slice/marge-bot.service
           └─11742 /usr/bin/docker run --name=marge -v /home/marge-bot/config/:/config -v /etc/localtime:/etc/localtime:ro smarkets/marge-bot --config-file=/config/bc-bot-config.yml

May 29 11:33:41 bc-web01d-vm systemd[1]: Starting marge-bot merge robot...
May 29 11:33:41 bc-web01d-vm docker[11676]: Using default tag: latest
May 29 11:33:43 bc-web01d-vm docker[11676]: latest: Pulling from smarkets/marge-bot
May 29 11:33:43 bc-web01d-vm docker[11676]: Digest: sha256: ...
May 29 11:33:43 bc-web01d-vm docker[11676]: Status: Image is up to date for smarkets/marge-bot:latest
May 29 11:33:43 bc-web01d-vm systemd[1]: Started marge-bot merge robot.

Troubleshooting

Keeping marge-bot up to date

Occasionally marge-bot can get stuck, and will need to be restarted. Also, the marge-bot code is under development, and we (currently) run the latest version of the Docker container. Therefore, it is sensible to restart the marge-bot service relatively regularly. This should be done via systemd to ensure that the Docker container is correctly removed.

Currently, we have a cron job in /etc/cron.daily like this:

#!/bin/sh

service marge-bot restart

There can be only one marge-bot

There should be only one instance of marge-bot running at any one time. However, docker containers are not automatically cleaned up. If you have only started marge-bot from the systemd configuration above, you should only have one marge-bot running, and its container should be called marge. You can check by running sudo docker -ps -a, and you should see something like:

$ sudo docker ps --all
CONTAINER ID        IMAGE                COMMAND                  CREATED             STATUS              PORTS               NAMES
f8a907df4d16        smarkets/marge-bot   "/bin/marge.app --co…"   45 minutes ago      Up 45 minutes                           marge

If you only want to see the container names, try sudo docker ps --all --format "table {{.Names}}".

If marge-bot has restarted itself, or you have created a number of containers whilst testing the bot, make sure to remove them with sudo docker rm -f NAME.

To automatically detect extra instances of marge-bot running, add a script like the one below to /etc/cron.hourly/:

#!/bin/bash

names=$(docker ps -all --filter ancestor=smarkets/marge-bot --format "table {{.Names}}" | tail -n +2 | grep -v marge)

for name in $names
do
    echo "Killing marge-bot Docker container $name"
    docker rm -f $name
done