Here is my fourth post about PatchBot.

In the first post I gave a short summary of how the system works and introduced JPCImporter, the first AutoPkg custom processor.

In the second post I introduced patch management and the second custom processor.

In the third post I showed the third custom processor and the code to run it at the right time.

In the first three blog posts I explained (in great detail) how my system, PatchBot, works.

Today I am going to cover how to take the pieces and put them together into a complete system.

Before we begin, there is now a page on Github that points to these four blog posts and three other repositories containing PatchBot and some example recipes. Keep an eye on it for news and improvements. Next thing on my roadmap is a set of scripts to send status messages to Slack instead of MS Teams.

A Few Requirements

The first thing you are going to need is a box. I had a Mac Mini spare and this is the perfect thing to shove in a server room to run our software. The load on the computer is tiny so anything will do. (I do all the development on a VM in Fusion on my MacBook Pro. You could do similar if you like.)

A fresh OS install with a single user, autopkg, is the next step.

Git

We will need git installed. The best way to get the latest version and easily keep it up to date is to use HomeBrew, so install it first, then brew install git. Installing Homebrew installs Apple’s XCode command line tools which includes a version of git, but this is an older version. I like to be up to date.

I assume that you have somewhere you can put a number of git repositories. In this example we will use private repos on GitHub but if you have access to something more secure then use it. You will do a lot of work writing recipes and it is good to have these all in version control.

The Terminal and Shell

I spend a lot of time setting up the shell environment on my machines. I now use zsh as my shell so I install oh-my-zsh, my favourite powerline fonts and turn on a number of plugins. Feel free to do as you prefer.

Software

The first thing to install is, of course, AutoPkg. Once you have it installed then get the default recipe repo and build a package.

autopkg repo-add recipes
autopkg make-override GoogleChrome.pkg
autopkg run GoogleChrome.pkg

AutoPkg Directories

If you now go to ~/Library/AutoPkg you should see three directories. Cache contains a cache for all the files that AutoPkg downloads and builds, your packages are in there. The second and third are RecipeOverrides and RecipeRepos which are obvious. We are going to add a fourth called Recipes which is searched by AutoPkg for recipes by default. The recipes we write will go in here along with the custom processors for PatchBot. So set that up.

mkdir -p ~/Library/AutoPkg/Recipes/Production
mkdir ~/Library/AutoPkg/Recipes/PatchManager
git clone https://github.com/Honestpuck/PatchBotProcessors.git \
~/Library/AutoPkg/Recipes/PatchBotProcessors

Python

AutoPkg comes with it’s own python 3 but this lacks the requests module that I use extensively. I could have used urllib instead but that’s more trouble and I wanted to avoid the work. To fix it we could add it to the python framework inside AutoPkg but that’s not recommended. Instead I chose to install the HomeBrew version of python 3 (once again, it’s more up to date than Apple’s) and add the module to it, so:

brew install python3
/usr/local/bin/pip3 install requests

Now we need to set the PYTHONPATH shell variable so the python inside AutoPkg can find our module.

echo "export PYTHONPATH=/usr/local/lib/python3.7/site-packages/" | \
tee -a ~/.zshenv >> ~/.bashrc

Check that is the correct path for you. Python may have changed. By the way, AutoPkg kindly places a link to the python inside itself at /usr/local/autopkg/python so we can run exactly the same python. I add the two lines below to .zshrc so I am always running the right python and the right version of pip.

echo "alias python3='/usr/local/autopkg/python'" | \
tee -a ~/.zshenv >> ~/.bashrc
echo "alias pip3='/usr/local/bin/pip3'" | \
tee -a ~/.zshenv >> ~/.bashrc

You will want to run source ~/.zshenv or source ~/bashrc depending on your shell. This performs the steps above to your currently open shell.

Our First Managed Package

Before we go to full automation let us put together all the pieces for one package and give it a test.

We already have a recipe for Google Chrome so let’s start with that.

I have a checklist of everything that needs to be done. I use checklists everywhere. My niece, a medical journalist, wrote about a meta study of the surgical use of checklists several years ago and for surgeons it’s a no-brainer with radically improved outcomes. I figure if it’s good enough for people much smarter than I am it’s a no-brainer for me too.

At this point it’s hard for us to actually test anything without a new package. I get around this by setting up everything by using a fake older version. Make a copy of the package you have just built and call it GoogleChrome_DELETE_ME.pkg and use it while setting up. One thing, when you attach the package to a version in the patch title attach it to an old version, not the latest as that should be empty ready for the real package. Just don’t enable any of the policies and you are safe.

Getting Automated

If every thing passed testing then we are ready to really get things rolling.

Make a list of every package that you need to manage and go through the checklist for the first half dozen so we have something to automate.

Grab a copy of my tools.

git clone https://github.com/Honestpuck/PatchBotTools.git \~/Documents/PatchBotTools

At this point I like to run `autopkg.sh` from the command line. It is easier than firing off our LaunchAgent. So run that and make sure that it does what you expect and you get the right packages and policies.

Now we need to test Move.py with the Production.py processor. The easiest way to do that is to go in to the “Test” patch policy of one of our titles in test and change the date in the self service description. It will contain a date, (2020-07-21) or similar. Change the 2020 to 2019 and you can guarantee that it is more than seven days ago. Now run Move.py and watch it trigger an autopkg run. Notice that Production.py does not check if it should run, that’s done by Move.py. This means that if you want to move a package into production early you can just run the .prod recipe yourself.

We need to install our LaunchAgents, I mentioned how to do that in the first post. Before you install it make sure the path to the script is correct, it assumes that we are running as the user `autopkg`.

# run in ~/Documents/PatchBotTools
mkdir /Users/"$(whoami)"/Library/LaunchAgents
/bin/cp autopkg.plist /Users/"$(whoami)"/Library/LaunchAgents/autopkg.plist
/bin/launchctl load /Users/"$(whoami)"/LaunchAgents/autopkg.plist
/bin/cp autopkg.plist /Users/"$(whoami)"/Library/LaunchAgents/move.plist
/bin/launchctl load /Users/"$(whoami)"/LaunchAgents/move.plist

As a side note, you can check that your LaunchAgents will run properly by firing them off yourself using launchctl

launchctl debug com.github.PatchBot.autopkg --stdout —stderr
launchctl kickstart com.github.PatchBot.autopkg

The first line sets the output of the next run of the LaunchAgent to your terminal and the second one fires the LaunchAgent as if the time had come for it to run.

Finishing Up

One final task. I like to have my recipe overrides and recipes in version control.

You can create a GitHub repo from the command line using `curl` but it’s fiddly and I can never remember how so go ahead and create the two we want in your browser before getting it working on your Mac like so, replacing <NAME> as required.

cd ~/Library/AutoPkg/Recipes
git init
git add *
git commit -m "first commit"
git remote add origin https://github.com/<NAME>/Recipes.git
git push -u origin master
cd ~/Library/AutoPkg/RecipeOverrides
git init
git add *
git commit -m "first commit"
git remote add origin https://github.com/<NAME>/RecipeOverides.git
git push -u origin master

You now have the system installed and running automatically. The next task is to continue adding more and more packages to your list. I find the hardest task is writing recipes for the rarer packages. Check out the AutoPkg documentation and Recipe Robot for help with writing recipes. Keep an eye on the logs and after a while turn the logging level down.

I hope you find my work useful. If something appears broken for you then, please, feel free to raise an issue on Github. I look forward to hearing from people using this work.