I constructed five of our jenkins agent AMI builds into an ansible_role-jenkins-[function]-agent piece, and then an aggregation jenkins pipeline build repo. The idea is – code from the role(s) are layered in the jenkins pipeline build repo. For example the base agent build (all the basic stuff we want on all of our agent AMIs) brings in the ansible_role-jenkins-base-agent role. The jenkins-maven-build-agent-ami pipeline job pulls in the base role, plus the ansible_role-jenkins-maven-agent role.
The issue comes in in actual development – this works great as long as the ansible_role code is stable. Once you need to edit the encompassing build, no problem – unless the role incorporates changes from a different fork and branch for the git submodule piece – you have to now adjust that by hand – and those changes will come through, so to build it later you now have to revert or change the commit id the git submodule is set to – not good. Adds complication and human error. If the code never changes – good. If the code could change or will change all the time – this isn’t a solution.
I’m trying a different approach, still using separate ansible role repos for specific constructs, but not using git submodule to pull them in. Instead there will be an installer.yml file, with specs like “gold master” – give me the rc code stored at the top level repo. “team”, for give me the team repo forks for these at the latest rc branch. “local” for my individual fork, or – specifying specific repo and version strings and crafting an isolated build. This would have sane defaults. This would be listed in the .gitignore, so it would not be dumped in and overwrite the installer.yml file. Instead there could be a cicd/install/ directory with sample and default installer.yml files – and scripting outside of that to adjust as needed. At build, these repos are brought in to the code on the fly. Something like that.
It looks like a better route for us for cicd than locking and indexing a git submodule into a repo and committing it and then having to adjust and commit and revert as you pull code through – just seems like that kind of constant git manipulation is just going to result in relying on humans. Thus bad.
Git submodules are awesome, to bring in code you are certain won’t be changing – say, a client tool that is used, but rarely changed, and not subject to internal development – that would be a perfect use. Not for code that is part of ongoing development for the project it gets pulled into.
This is all going to be open sourced it looks like, so this whole product should come together at some point in a publicly accessible repo. It just looks like git submodules are useful in particular circumstances, but not where the code is going to be a moving target.
— doug