Bash Rocks (Direct Access to System) - Intuitive Engineering

As a basic tool, bash rocks.

In the late 1990’s, perl ruled. It was a completely flexible non-structured and not object oriented at all (though a virtual object oriented approach was possible). Perl ran everything. It slid in when java was cumbersome and slow and C/C++ was tripping over it’s own memory blocks all over the place. I wrote a firewall configuring iptables using perl and it was the go to tooling for basic scripting.

Good riddance. You could write good perl, but you could also make it so obscure (there was a contest for that, actually, the Obfuscated Perl Contest…) that it was unreadable.

Python is our language of choice now. The only issue with python is CentOS 7 being stuck at python 2.7, and much of what we do requires python 3.6, and unfortunately, Redhat used python 2.7 in yum of all things. So you can’t upgrade the whole server, you can only install 3.6 next to 2.7.

Ansible is my choice for anything to do with configuration or changing a server but – but it has limits, and these show up particularly when you are manipulating configuration files in Jenkins.

Jenkins configs are xml. In some cases they repeat. While ansible has modules to directly manipulate line by line, or a block of text at a time it fails to have the concept of “global”, for repetitive changes repeating, or of both before and after to locate lines. There is an “insertbefore” and an “insertafter” for the module lineinfile, but only one or the other can be used. In other words you can’t tell it to change a line that you identify after one string but before another. In a series of lines, all of which need to change, ansible changes the last. Or the first.

Bash on the other hand… This script below adjusts the controller’s numExecutors key:value, ignoring and skipping the agents (slaves) numExecutors. This was the first piece of tooling for allowing configuration of (1) does the box have agents configured or not?, (2) how many executors (processes that accept builds in Jenkins) are configured on the master?, and (3) if there are agents, how many executors does each of those get?

#! /bin/bash

# ansible can't parse this for shit so, bash it is...

# check for arg
# arg is the number of master executors to be configured
if [ -z "$1" ]
  then
    echo "No argument supplied"
    exit 1
fi

# config file
CONFDIR="/var/lib/jenkins"

# read config.xml into an array...
declare -a configarray
#readarray configarray .*$ ]]; then
        echo "triggering change mode at version line..."
        # set change mode on
        CHANGE_MODE=true
        echo $CURRENT >> $TMPCONF
    elif [[ $CURRENT =~ ^.*\.*\$ ]]; then
        if [[ $CHANGE_MODE == "true" ]]; then
            echo "found master numExecutor statement..."
            # turn off change mode
            CHANGE_MODE=false
            # finish by replacing numExecutors with $1
            echo "  $1" >> $TMPCONF
        else
            # keep the line if not to be changed
            echo "found NON master numExecutors line:  $CURRENT"
            echo $CURRENT >> $TMPCONF
        fi
    else
        # we keep these lines
        echo $CURRENT >> $TMPCONF
    fi
done

# backup the config.xml to tmp...
DATE=`date +%Y%m%d%H%M%S`
mkdir -p /tmp/old_configs

cp $CONFDIR/config.xml /tmp/old_configs/config.xml.$DATE
cp $TMPCONF $CONFDIR/config.xml

# reload the config
/var/lib/jenkins/scripts/reload-config.sh

What happens is the installer module looks at the variables in installer-config.yml for

orion_use_agents
orion_master_executors
orion_agent_executors

If these are set, the installer.yml drops yaml variable files pushing those key:value pairs into the ansible role to be acted on later, when packer calls the standalone ansilbe role to construct the finishing touches on Nebula-in-a-Box AMIs. If they are not set the distributed Nebula config.xml for Jenkins remains, with slaves and executors left alone.

The last line calls another script (reload-config.sh) which uses the jenkins-cli.jar to call the reload configuration method, and instead of restarting Jenkins, just ask it to re-read the config into memory. Takes about 1 second, and does not interrupt Jenkins.

This will be re-written in python with test coverage in the next several months. For an ad hoc tool, though, to create POC and raw methods, bash rocks.

— doug