How to set up marge-bot¶
marge-bot is an automated service that merges GitLab merge requests. When a merge-request is ready, just assign it to its user, and Marge will do all any necessary rebase-wait-retry for you. If anything goes wrong (merge conflicts, tests that fail, etc.) Marge will leave a comment on the merge-request.
Install docker¶
On the server that will run marge-bot, install Docker:
sudo apt-get update
sudo apt-get install docker.io
Create a non-privileged user¶
Create a new user, with a home directory, and add the new user to the docker group:
sudo adduser --disabled-password marge-bot
sudo usermod -a -G docker marge-bot
Create credentials¶
To identify the instance of marge-bot on GitLab you will need to create some private credentials.
SSH key¶
On the server that will be running the bot, create a new SSH key (replace the email address with a valid address):
sudo su marge-bot
cd
ssh-keygen -t ed25519 -C marge-bot@invalid -f marge-bot-ssh-key -P ''
You will need both the public and private keys in later steps.
GitLab personal access token¶
Go to the personal access tokens page in the settings of your GitLab user account (probably @bc-bot).
Create a new token with api level access, and store the token somewhere safe (do not close the browser window until you are finished).
This token gives @bc-bot access to the GitLab API for all the groups and projects to which the bot user belongs, so that it can see which merge requests are assigned to @bc-bot.
Upload the public SSH key to GitLab¶
Go to the ssh keys page in the settings of your GitLab user account (probably @bc-bot).
Add the new public key to the list of keys for the bot, and give it a name that makes it clear that the key will be used by marge-bot, and which server will be used.
Create a marge-bot configuration¶
Create a directory to hold a configuration file:
sudo su marge-bot
cd
mkdir config
Place the new personal access token in a file called /home/marge-bot/config/token.
Place the private SSH key in a file called /home/marge-bot/config/marge-bot-ssh-key.
Create a file called /home/marge-bot/config/bc-bot-config.yml and add in a basic configuration:
# File /home/marge-bot/config/bc-bot-config.yml
debug: false
add-part-of: true
# add-reviewers requires a self-hosting GitLab instance.
add-reviewers: false
add-tested: true
# chose one way of specifying the Auth token
# auth-token:
auth-token-file: /config/token
branch-regexp: .*
ci-timeout: 60min
embargo: Friday 6pm - Monday 8am
batch: false
git-timeout: 600s
gitlab-url: "https://gitlab.com"
# impersonate-approvers requires a self-hosting GitLab instance.
impersonate-approvers: false
project-regexp: .*
# chose one way of specifying the SSH key
ssh-key-file: /config/marge-bot-ssh-key
# ssh-key:
Note that we use files to store the PAT and SSH key, which are mounted in the Docker volume.
This is to avoid leaking secrets to other users on the server who can run docker ps.
The front page of the marge-bot GitHub page has more documentation on configuring the bot.
Test marge-bot¶
Check that marge-bot runs correctly in docker:
docker run -v /home/marge-bot/config/:/config \
-v /etc/localtime:/etc/localtime:ro \
smarkets/marge-bot \
--config-file=/config/bc-bot-config.yml
Note that -v /etc/localtime:/etc/localtime:ro mounts the localtime file as read-only, which ensures that marge-bot always runs in a Docker container with the same timezone as the host server.
Don't worry if marge-bot takes a while to start, but once it does you should see a stream of output like this:
Mar 09 17:53:52 bc-web01d-vm docker[24526]: 2019-03-09 17:53:52,542 INFO Nothing to merge at this point...
Mar 09 17:53:53 bc-web01d-vm docker[24526]: 2019-03-09 17:53:53,543 INFO Fetching merge requests assigned to me in beautifulcanoe/projects/actss...
Mar 09 17:53:54 bc-web01d-vm docker[24526]: 2019-03-09 17:53:54,433 INFO Nothing to merge at this point...
Use Ctrl+c to stop the bot running before moving to the next stage.
Create a systemd service¶
Once marge-bot works, you will want to run it as a background daemon that starts automatically when the server reboots.
Create a new file called /etc/systemd/system/marge-bot.service to hold the systemd configuration.
Your new service will need to run as the marge-bot user and group, after docker.service has started.
You will also need to include an [Install] section in the configuration, in order to enable the service to start on boot.
Since marge-bot is running in Docker, it is helpful to give the container a name (i.e. --name=marge) so that it is easier to stop or rm the container via systemd.
For example:
[Unit]
Description=marge-bot merge robot
ConditionPathExists=/home/marge-bot/config
After=docker.service
Requires=docker.socket
[Service]
User=marge-bot
Group=marge-bot
ExecStartPre=/usr/bin/docker pull smarkets/marge-bot
ExecStart=/usr/bin/docker run --name=marge -v /home/marge-bot/config/:/config -v /etc/localtime:/etc/localtime:ro smarkets/marge-bot --config-file=/config/bc-bot-config.yml
ExecStop=/usr/bin/docker stop marge
ExecStopPost=/usr/bin/docker rm -f marge
KillMode=process
Restart=on-failure
RuntimeDirectory=/home/marge-bot/
RuntimeDirectoryMode=0755
TimeoutSec=infinity
[Install]
WantedBy=multi-user.target
Alias=marge-bot.service
Enable and start the bot:
sudo systemctl enable marge-bot.service
sudo systemctl start marge-bot.service
and check that the bot is running:
$ sudo systemctl status marge-bot
● marge-bot.service - marge-bot merge robot
Loaded: loaded (/etc/systemd/system/marge-bot.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2019-05-29 11:33:43 BST; 2s ago
Process: 11668 ExecStopPost=/usr/bin/docker rm -f marge (code=exited, status=0/SUCCESS)
Process: 11340 ExecStop=/usr/bin/docker stop marge (code=exited, status=0/SUCCESS)
Process: 11676 ExecStartPre=/usr/bin/docker pull smarkets/marge-bot (code=exited, status=0/SUCCESS)
Main PID: 11742 (docker)
Tasks: 10 (limit: 4662)
CGroup: /system.slice/marge-bot.service
└─11742 /usr/bin/docker run --name=marge -v /home/marge-bot/config/:/config -v /etc/localtime:/etc/localtime:ro smarkets/marge-bot --config-file=/config/bc-bot-config.yml
May 29 11:33:41 bc-web01d-vm systemd[1]: Starting marge-bot merge robot...
May 29 11:33:41 bc-web01d-vm docker[11676]: Using default tag: latest
May 29 11:33:43 bc-web01d-vm docker[11676]: latest: Pulling from smarkets/marge-bot
May 29 11:33:43 bc-web01d-vm docker[11676]: Digest: sha256: ...
May 29 11:33:43 bc-web01d-vm docker[11676]: Status: Image is up to date for smarkets/marge-bot:latest
May 29 11:33:43 bc-web01d-vm systemd[1]: Started marge-bot merge robot.
Troubleshooting¶
Keeping marge-bot up to date¶
Occasionally marge-bot can get stuck, and will need to be restarted.
Also, the marge-bot code is under development, and we (currently) run the latest version of the Docker container.
Therefore, it is sensible to restart the marge-bot service relatively regularly.
This should be done via systemd to ensure that the Docker container is correctly removed.
Currently, we have a cron job in /etc/cron.daily like this:
#!/bin/sh
service marge-bot restart
There can be only one marge-bot¶
There should be only one instance of marge-bot running at any one time.
However, docker containers are not automatically cleaned up.
If you have only started marge-bot from the systemd configuration above, you should only have one marge-bot running, and its container should be called marge.
You can check by running sudo docker -ps -a, and you should see something like:
$ sudo docker ps --all
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
f8a907df4d16 smarkets/marge-bot "/bin/marge.app --co..." 45 minutes ago Up 45 minutes marge
If you only want to see the container names, try sudo docker ps --all --format "table {{.Names}}".
If marge-bot has restarted itself, or you have created a number of containers whilst testing the bot, make sure to remove them with sudo docker rm -f NAME.
To automatically detect extra instances of marge-bot running, add a script like the one below to /etc/cron.hourly/:
#!/bin/bash
names=$(docker ps -all --filter ancestor=smarkets/marge-bot --format "table {{.Names}}" | tail -n +2 | grep -v marge)
for name in $names
do
echo "Killing marge-bot Docker container $name"
docker rm -f $name
done