As a basic tool, bash rocks.
In the late 1990’s, perl ruled. It was a completely flexible non-structured and not object oriented at all (though a virtual object oriented approach was possible). Perl ran everything. It slid in when java was cumbersome and slow and C/C++ was tripping over it’s own memory blocks all over the place. I wrote a firewall configuring iptables using perl and it was the go to tooling for basic scripting.
Good riddance. You could write good perl, but you could also make it so obscure (there was a contest for that, actually, the Obfuscated Perl Contest…) that it was unreadable.
Python is our language of choice now. The only issue with python is CentOS 7 being stuck at python 2.7, and much of what we do requires python 3.6, and unfortunately, Redhat used python 2.7 in yum of all things. So you can’t upgrade the whole server, you can only install 3.6 next to 2.7.
Ansible is my choice for anything to do with configuration or changing a server but – but it has limits, and these show up particularly when you are manipulating configuration files in Jenkins.
Jenkins configs are xml. In some cases they repeat. While ansible has modules to directly manipulate line by line, or a block of text at a time it fails to have the concept of “global”, for repetitive changes repeating, or of both before and after to locate lines. There is an “insertbefore” and an “insertafter” for the module lineinfile, but only one or the other can be used. In other words you can’t tell it to change a line that you identify after one string but before another. In a series of lines, all of which need to change, ansible changes the last. Or the first.
Bash on the other hand… This script below adjusts the controller’s numExecutors key:value, ignoring and skipping the agents (slaves) numExecutors. This was the first piece of tooling for allowing configuration of (1) does the box have agents configured or not?, (2) how many executors (processes that accept builds in Jenkins) are configured on the master?, and (3) if there are agents, how many executors does each of those get?
#! /bin/bash
# ansible can't parse this for shit so, bash it is...
# check for arg
# arg is the number of master executors to be configured
if [ -z "$1" ]
then
echo "No argument supplied"
exit 1
fi
# config file
CONFDIR="/var/lib/jenkins"
# read config.xml into an array...
declare -a configarray
#readarray configarray < /var/lib/jenkins/config.xml # Include newline.
IFS=''
readarray -t configarray < $CONFDIR/config.xml # Exclude newline.
# tmp config
TMPCONF="/tmp/tmp_master_executors_config.xml"
# remove if exists
rm -f ${TMPCONF}
# go find and edit the lines before each tag...
CHANGE_MODE=false
for CURRENT in "${configarray[@]}"
do
#echo "$CURRENT"
if [[ $CURRENT =~ ^.*\.*$ ]]; then
echo "triggering change mode at version line..."
# set change mode on
CHANGE_MODE=true
echo $CURRENT >> $TMPCONF
elif [[ $CURRENT =~ ^.*\.*\ $ ]]; then
if [[ $CHANGE_MODE == "true" ]]; then
echo "found master numExecutor statement..."
# turn off change mode
CHANGE_MODE=false
# finish by replacing numExecutors with $1
echo " $1 " >> $TMPCONF
else
# keep the line if not to be changed
echo "found NON master numExecutors line: $CURRENT"
echo $CURRENT >> $TMPCONF
fi
else
# we keep these lines
echo $CURRENT >> $TMPCONF
fi
done
# backup the config.xml to tmp...
DATE=`date +%Y%m%d%H%M%S`
mkdir -p /tmp/old_configs
cp $CONFDIR/config.xml /tmp/old_configs/config.xml.$DATE
cp $TMPCONF $CONFDIR/config.xml
# reload the config
/var/lib/jenkins/scripts/reload-config.sh
What happens is the installer module looks at the variables in installer-config.yml for
- orion_use_agents
- orion_master_executors
- orion_agent_executors
If these are set, the installer.yml drops yaml variable files pushing those key:value pairs into the ansible role to be acted on later, when packer calls the standalone ansilbe role to construct the finishing touches on Nebula-in-a-Box AMIs. If they are not set the distributed Nebula config.xml for Jenkins remains, with slaves and executors left alone.
The last line calls another script (reload-config.sh) which uses the jenkins-cli.jar to call the reload configuration method, and instead of restarting Jenkins, just ask it to re-read the config into memory. Takes about 1 second, and does not interrupt Jenkins.
This will be re-written in python with test coverage in the next several months. For an ad hoc tool, though, to create POC and raw methods, bash rocks.
— doug