July | 2020 | Ramblings of an Eggplant

To help with restores or migration to another piece of hardware I find it useful to keep track of which packages are installed on my Linux systems. This isn’t very difficult, but it has taken a bit of experimenting to come up with with something that works fairly well and is automated.

Create in /etc/systemd/system/status-email-root@.service. This is a helper service for sending emails on service failures.

[Unit]
Description=status email for %i to Jon

[Service]
Type=oneshot
ExecStart=/usr/local/sbin/systemd-email.sh root %i

/usr/local/sbin/systemd-email.sh looks like this

#!/bin/sh

debug() { ! "${log_debug-false}" || log "DEBUG: $*" >&2; }
log() { printf '%s\n' "$*"; }
warn() { log "WARNING: $*" >&2; }
error() { log "ERROR: $*" >&2; }
fatal() { error "$*"; exit 1; }
try() { "$@" || fatal "'$@' failed"; }

mydir=$(cd "$(dirname "$0")" && pwd -L) || fatal "Unable to determine script directory"


/usr/sbin/sendmail -t <<ERRMAIL
To: $1
From: systemd <root@$HOSTNAME>
Subject: $2
Content-Transfer-Encoding: 8bit
Content-Type: text/plain; charset=UTF-8

$(systemctl status --full "$2")
ERRMAIL

Create in /etc/systemd/system/track-installed-packages.service with the content. Note the “showmanual” argument to “apt-mark”. This makes sure to only output the packages that were manually installed and not include those that were installed as dependencies. This keeps the list short and if dependencies change later keeps from installed unneeded packages later.

[Unit]
Description=track installed packages
OnFailure=status-email-root@%n.service

[Service]
Type=oneshot
ExecStart=/usr/bin/apt-mark showmanual
StandardOutput=file:/home/installed-packages.txt

Now create /etc/systemd/system/track-installed-packages.timer

[Unit]
Description=track installed packages daily

[Timer]
OnCalendar=daily
Persistent=true

[Install]
WantedBy=timers.target

Now enable the timer with

systemctl daemon-reload
systemctl enable track-installed-packages.timer
systemctl start track-installed-packages.timer

Now if you backup /home, you’ll always have the most recent list of installed packages.

I have a similar pair of systemd files for snaps that calls “snap list” and outputs to “/home/snap-packages.txt”. Snap doesn’t appear to have an option to specify only manually installed snaps. This may also error out if there are no snaps installed.

I have been setting up a few Jenkins nodes lately and decided that I should write up the configuration that I’m using to share with others.

Create the node in Jenkins

The first thing to do is to create the node in Jenkins. Start by logging into your Jenkins host, then visit the “Manage Jenkins” link. Once there, visit “Manage Nodes” and then click “New Node” on the left.

Give your node a name. It’s a good idea to avoid spaces and special characters. I use letters, numbers, underscores and hyphens. Select “Permanent Agent” and then “OK”.

Here you need to specify the working directory, labels and the usage. I usually set the usage to only build jobs with a matching label expression. This is useful when setting up nodes per job to make sure that the node doesn’t get used for other random jobs. You may also want to specify an email address to notify when the node goes online and/or offline.

Once you have saved the configuration you will see a page specifying that the agent is offline and how to launch it. The important piece of information here is the secret. This will be a very long string of letters and numbers.

Linux Setup

First create a user in Linux that the node will run as. This user should not have any special privileges.

sudo adduser JENKINS_BUILD_USER

Replace “JENKINS_BUILD_USER” with the username that you are using. By default this user has a locked password so no one can login as this user.

In “/home/JENKINS_BUILD_USER” create the file “start-jenkins-node.sh” to start the node

#!/bin/sh

debug() { ! "${log_debug-false}" || log "DEBUG: $*" >&2; }
log() { printf '%s\n' "$*"; }
warn() { log "WARNING: $*" >&2; }
error() { log "ERROR: $*" >&2; }
fatal() { error "$*"; exit 1; }
try() { "$@" || fatal "'$@' failed"; }

mydir=$(cd "$(dirname "$0")" && pwd -L) || fatal "Unable to determine script directory"

jenkins_host=JENKINS_HOST
jenkins_node_name=NODE_NAME
jenkins_node_secret=SECRET

cd "${mydir}"
# --no-check-certificate is needed if the certificate store does not recognize the jenkins host certificate
try wget https://${jenkins_host}/jnlpJars/agent.jar -O agent.jar

# -noCertificateCheck is needed if the certificate isn't recognized
nohup java -jar agent.jar -jnlpUrl https://${jenkins_host}/computer/${jenkins_node_name}/slave-agent.jnlp -secret ${jenkins_node_secret} -workDir "${HOME}" > "${HOME}"/jenkins-node.log 2>&1

Replace JENKINS_HOST with the hostname that Jenkins is running on. This script assumes that Jenkins is running at hte root of your server. If that’s not the case you’ll want to append the base path to the end of JENKINS_HOST. Replace NODE_NAME with the name of the node and SECRET with the secret from the node configuration on the Jenkins host.

Mark the file executable.

chmod +x /home/JENKINS_BUILD_USER/start-jenkins-node.sh

Create “/etc/systemd/system/jenkins_node.service”

[Service]
Type=simple
ExecStart=/home/JENKINS_BUILD_USER/start-jenkins-node.sh
WorkingDirectory=/home/JENKINS_BUILD_USER
Restart=always
RestartSec=60
User=JENKINS_BUILD_USER

[Unit]
After=network-online.target
Wants=network-online.target

[Install]
WantedBy=default.target

Replace JENKINS_BUILD_USER with the user that you created. Then you can enable and start the service with

sudo systemctl daemon-reload
sudo systemctl enable jenkins_node
sudo systemctl start jenkins_node

At this point you should see your node online in Jenkins and you are ready to use it for jobs.

Ramblings of an Eggplant

Monthly Archives: July 2020

Tracking packages installed

Configure Linux Jenkins node

Create the node in Jenkins

Linux Setup