The Docker Engine is actually built on top of a few Linux technologies, including Kernel namespaces and control groups (cgroups), and various filesystem layers. They work together to isolate what’s in the container from your computer, whilst sharing the common pieces, to avoid duplication.

But the Mac does not use the Linux kernel, it uses the Darwin hybrid kernel. So how can we run technology built for Linux on a different Kernel?

Virtualisation

The answer is virtualisation, where we create a virtual version of our computer at the hardware level, but run a different operating system inside it, in this case Linux!

When you install Docker Desktop on a Mac, this is exactly what is happening behind the scenes:

• A Linux virtual machine (VM) is created behind the scenes, and the Docker Engine is installed inside it.
• Docker command line tools are installed onto your Mac, and configured to talk to the Docker Engine inside the Linux VM.

In this way, Docker Desktop provides a turnkey solution for running Docker on a Mac.

Older versions of Docker Desktop used a technology called HyperKit to manage the VM, but have more recently transitioned to Apple’s Virtualization Framework which provides greater performance, thanks to support for the Virtual I/O Device specification.

Colima

Recent licensing changes made it quite a bit more expensive for businesses to use Docker Desktop, but there are free and open source alternatives.

Colima is a free and open source project which handles running Docker Engine inside a lightweight Alpine Linux virtual machine. It is a little more work to get started, but once you’re up and running it acts like a drop-in replacement, compatible with all the same Docker commands you are used to.

Installing

You will need to make sure you have Homebrew installed first. Next, quit Docker Desktop, then run the following commands:

# Install Colima and the Docker command line tools:
brew install colima docker docker-compose docker-buildx

# Enable the Compose and BuildKit plugins:
mkdir -p ~/.docker/cli-plugins
ln -sfn $(brew --prefix)/opt/docker-compose/bin/docker-compose ~/.docker/cli-plugins/docker-compose
ln -sfn $(brew --prefix)/opt/docker-buildx/bin/docker-buildx ~/.docker/cli-plugins/docker-buildx

# Add the following to your zsh/bash profile, so Docker can find Colima:
# (don't forget to reload your shell)
export DOCKER_HOST="unix://${HOME}/.colima/default/docker.sock"

Now you’re ready to create the virtual machine where the Docker Engine will run, using the correct command for your version of macOS:

• macOS 13 “Ventura” – uses macOS Virtualisation Framework with Virtiofs for the best possible performance:$ colima start --vm-type vz --mount-type virtiofs
• macOS 12 “Monterey” or older – uses QEMU with SSHFS$ colima start

Colima will use 2 vCPUs and 2GB RAM by default, but if you run a lot of containers at once, you may need to adjust that. For example, to double the resources, add --cpu 4 --memory 4 to the colima start command you used above.

You can now verify everything is running properly:

$ colima status
INFO[0000] colima is running using macOS Virtualization.Framework
INFO[0000] arch: aarch64
INFO[0000] runtime: docker
INFO[0000] mountType: virtiofs
INFO[0000] socket: unix:///Users/ryan/.colima/default/docker.sock

$ docker system info | grep Operating
  Operating System: Alpine Linux v3.16   # if it says Docker Desktop here then something is wrong

You should now be able to use docker and docker compose commands normally, they should all Just Work transparently.

When you next reboot your Mac, you will need to remember to run colima start to bring the virtual machine back up, before you can use Docker again.

Importing your existing data?

Now that you have Colima up and running, you’ll have noticed that things look pretty sparse. This is because all the existing container images and volumes are still within Docker Desktop’s managed VM.

If this data is important to you, Docker provides a way to backup and restore containers, and also a way to migrate data volumes.

I decided to skip this step as my own use of Docker is for local development. Any data I have in Docker is regenerated automatically when I run my projects.

Cleanup

Assuming you got this far and you’re happy with Colima, you will probably want to send your old Docker environment to the Trash, otherwise the containers and volumes will stick around consuming valuable disk space.

You can use the following commands to tidy everything up:

# Remove old Docker data:
rm -rf /Users/${USER}/Library/Containers/com.docker.docker

# Remove the old Docker Desktop application:
# Note: If you installed it via Homebrew as a Cask,
# run `brew remove --cask docker` instead.
rm -rf /Applications/Docker.app

# Remove the old Docker configuration files:
sudo rm -rf /usr/local/lib/docker
rm -rf ~/.docker

Enjoy!

I continued making improvements to only show the relevant information at the right time, for example hiding the Git branch if I wasn’t currently in a Git repository. and the same with the language version. This stopped the prompt becoming too large and cumbersome.

Problems Arise

Over the years I added more code to this custom prompt, as new technologies were added to my toolbox, however problems started to show.

This custom prompt code isn’t just executed when you open a new tab, it’s run every time the prompt has to display. So the amount of code you put in there has an impact on how quickly your prompt will display. Over the years, it had become so bloated that it was now taking several seconds to execute, and remember this was happening every time the prompt needed to be displayed.

Let’s just say this Bash script had evolved very … organically … and making changes to it was becoming more difficult. Some of the more “artisanal” parts from the early days had become almost undecipherable.

Refactor

My initial fixes were to go through the oldest parts of the prompt code involved janky functions for setting custom colours, and were needlessly defining variables all over the place. Some other areas were invoking commands in subshells and capturing the output, in order to use it later, when in reality they always returned the same value, and were just wasting cycles.

I ripped all of this out to reduce the amount of code being executed, removing code defining colours I never used, and replacing subshells with constants defined at the top of the script. I also looked into the performance of things like rbenv and pyenv to see if they could be sped up.

Performance got a lot better, but it was still taking around a second or so. Satisfied for the time being, I made a note to look into alternatives in the near future.

Board the Starship

A friend of mine had previously recommended a cross-shell prompt called Starship which is written in Rust, but my initial experiments hadn’t been very successful. Given Rust programs have a reputation for being blazing fast, I thought it was time to see how the project had progressed, and maybe give it another try.

I’m happy to say this time, I was able to get Starship to do everything I wanted.

Install

I followed the official documentation to install Starship via Homebrew:

brew install starship

I did not opt to install their “Nerd Font” as this is only needed if you want to make use of the various icons and logos in your prompt, and I wanted to keep things simple.

Next step was to disable my old custom prompt code, and replace it with this:

export STARSHIP_CONFIG=~/.starship.toml
eval "$(starship init bash)"

Notice I have set a custom configuration file location, this is purely because I already have a system for synchronising my config files between systems, and the default location wasn’t suitable. You can probably skip that step.

Restarting the shell I was greeted with my new super fast prompt!

Configuration

The default prompt configuration tries to cover all the bases, but I found it too verbose, so I took a look at the configuration options.

Firstly, I wanted to limit what was displayed to a few key pieces of information, rather than have every supported option jostling for position. I wanted to have the prompt on a single line, and show only the current working directory, Ruby and Python information, the current Git branch, and finally the prompt character:

# ~/.starship.toml
add_newline = false
format = """
$directory\
$ruby\
$python\
$git_branch\
$character
"""

Next I wanted to customise the sections I had enabled so they didn’t show a symbol, and were all bracketed, so each section would be distinct, yet compact. I started by using the Starship presets and then made my modifications on top, resulting in this extra config:

# ~/.starship.toml
[git_branch]
format = '\[[$symbol$branch]($style)\]'
style = 'bold white'
symbol = ''

[python]
format = '\[[${symbol}${pyenv_prefix}(${version})(\($virtualenv\))]($style)\]'
symbol = ''

[ruby]
format = '\[[$symbol($version)]($style)\]'
symbol = ''

Finally I wanted to make the directory section better match the others, by bracketing it, and making it show the full file path for clarity:

# ~/.starship.toml
[directory]
truncation_length = 0
truncate_to_repo = false
format = '\[[$path]($style)[$read_only]($read_only_style)\]'

You can see all the options for the prompt sections I have used below:

• directory
• ruby
• python
• git_branch

Result

The end result looks exactly the same as my old custom prompt did, which is a testament to the customisability of Starship. The performance difference is striking. Bash now starts almost instantly, and my prompt returns so quickly it’s almost imperceptible.

I’m a big proponent of making time to address seemingly small issues like these. They have a habit of building up over time, until every interaction you’re having with your computer is like drying yourself with sandpaper.

I’m very happy I took another look at Starship, and can finally tick this off my todo list.

At work last year I redid some internal documentation for a large Ansible automation project and had a few things in mind:

1. The documentation for the code should live with that same code to make it easy to keep up to date as the code changes.
2. The documentation should be as lightweight as possible, avoiding the need to maintain any hosting software or databases.
3. The documentation should be easy to write so it doesn’t feel like a chore.

These requirements made me immediately think of Markdown. We could commit Markdown alongside the code itself, write it easily, use any editor, read it locally and so on.

Paired with a static site generator, you can actually get some really nice looking documentation with a minimum of effort. The generator takes the Markdown files, and creates static HTML pages using a template.

For the work project we ended up using software called MkDocs which is very good, but focussed around project documentation; for this site I needed to look at something more focussed on blogging. I had already decided I wanted to try GitHub Pages so that publishing a new post would be as easy as committing to a repository, and the new post would be generated and made available almost right away.

Jekyll is a static site generator that is supported by GitHub Pages out of the box and is written in Ruby, which I have experience with, so it felt like a good place to start.

I followed the quick-start instructions on the Jekyll website and was up and running pretty quickly, but I’m a tinkerer, I like to dig deeper into things and see how they work, so I found myself cloning the default theme to see exactly how it built a page and how I could customise it.

Most of the customisations I made were small, things like adding the estimated reading time to each post, displaying the categories the post belongs to, adding ARIA tags for accessibility and so on.

Committing all of this to a GitHub repository and turning GitHub Pages on in the repo settings was painless and everything worked as expected, hurray!

My next issue was that I wanted to add archive pages to list posts by category or date, so that I wouldn’t need to add this later once I actually had some content. Going back to the earlier goal of simplicity I wanted the tool to handle generating archive pages for me, but out of the box Jekyll can’t do this. It is however extensible through plugins, and I was able to find and install a plugin called jekyll-archives, but here I hit a snag; this plugin isn’t supported by the version of Jekyll that is used by GitHub Pages.

What to do? Well luckily for me, GitHub have recently launched their own CI/CD automation workflow called GitHub Actions, allowing you to perform certain actions when code is committed to a repository, so we have an option to run Jekyll directly instead of relying on the one integrated with GitHub Pages. This allows us to do anything you could do normally with Jekyll, including use custom plugins.

Building a workflow

GitHub Pages can be configured to skip running Jekyll itself and instead just take an existing set of static HTML files from a branch named gh-pages and use those, so I needed to setup a workflow for GitHub Actions that would do the following:

• Checkout the latest revision of the repository
• Set up a Ruby environment and install Jekyll
• Run the jekyll build command
• Push the result of the build command to the gh-pages branch

To create a GitHub Action you define the workflow in YAML format and commit that file into a .github/workflows/ folder in your repository, after which you’ll see it listed under the Actions tab of your repository. So let’s translate each step above into an action in our workflow, using the actions available.

Workflow start

We want to start by naming the workfow and configuring it to run any time commits are pushed to the master branch, we also configure the workflow to be run inside an Ubuntu linux environment which is setup fresh each time the workflow runs. We define the steps key which we will fill in as we go.

---
name: "Build Jekyll and push to 'gh-pages' branch"
on:
  push:
    branches:
      - master

jobs:
  build:
    runs-on: ubuntu-latest
    steps:

Checkout the code

The first item to add to steps is an action to checkout the latest revision of the master branch, which is done like so:

- name: Checkout
  uses: actions/checkout@v2

Ruby

Next we want to setup a Ruby environment and there is an action that can handle that for us:

- name: Set up Ruby
  uses: actions/setup-ruby@v1
  with:
    ruby-version: 2.6

Now we should have a Ruby environment and our Jekyll project checked out ready to go, we need to install Jekyll and it’s dependencies from the Gemfile in our project. Here we don’t need any particular action, we just want to run a command:

- name: Bundle install
  run: |
    bundle install

Jekyll

Next we run Jekyll’s build command against our project and tell it to place the resulting static HTML files in a folder named:

- name: Bundle install
  run: |
    bundle exec jekyll build -d target

Now we should have the static content ready to commit to the gh-pages branch, but I wasn’t sure how to proceed, I came across a blog post by Benjamin Lannon which shows that you can run git commands normally, however I wanted to commit to a different branch (gh-pages in this case) and only commit the result of the jekyll build command, to avoid accidentally including non-content, such as the Gemfile. After a bit of Googling I found this very helpful Gist which showed a very tidy way of avoiding this issue entirely.

Using the subtree split feature of Git we can split the repository apart by taking only the commits that affected our target folder created in the previous step. Since this will be a single commit containing the entire generated site, we then force push that to the desired branch for GitHub Pages to pick up and deploy for us.

Git config

Let’s configure a user for the commits to be made as:

- name: Setup Git config
  run: |
    git config user.name "GitHub Actions Bot"
    git config user.email "<>"

Next we create the commit using git subtree split:

- name: Commit
    run: |
      git add target
      git commit -m "$(git log -1 --pretty=%B)"
      git subtree split --prefix target -b gh-pages
      git push -f origin gh-pages:gh-pages

That’s it! Now we should be able to use the full functionality of Jekyll and any plugins we like, but we still get the simplicity of pushing a single commit to our repository, in order to publish new content. There are a couple of enhancements we can make to speed up the performane of this workflow though.

Caching

Rather than setting up the Ruby environment from scratch every time, we want to cache the gems we’re using so that they can be installed much quicker next time. There’s a Cache action we can use to do this for a variety of languages including Ruby.

First we need to include the Cache action in our workflow, place this new task above the Set up Ruby task:

- uses: actions/cache@v2
  with:
    path: vendor/bundle
    key: ${{ runner.os }}-gems-${{ hashFiles('**/Gemfile.lock') }}
    restore-keys: |
            ${{ runner.os }}-gems-

This will instruct GitHub Actions to cache the data in vendor/bundle using a specific key, in this case the key includes the operating system of the system running our workflow, and the hashFiles function is used to generate a hash for a given file, in this case our Gemfile.lock, so that we only rebuild our cache if the Gemfile (and therefore the hash) changes.

Next we need to configure Bundler to actually use this cache when installing gems. Modify the Bundle install task to look like this:

- name: Bundle install
  run: |
    bundle config path vendor/bundle
    bundle install

Summary

Putting it all together, here is the here is the complete workflow configuration we just built:

---
name: "Build Jekyll and push to 'gh-pages' branch"
on:
  push:
    branches:
      - master

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Set up Ruby
        uses: actions/setup-ruby@v1
        with:
          ruby-version: 2.6

      - name: Bundle install
        run: |
          bundle install

      - name: Bundle install
        run: |
          bundle exec jekyll build -d target

      - name: Setup Git config
        run: |
          git config user.name "GitHub Actions Bot"
          git config user.email "<>"

      - name: Commit
          run: |
            git add target
            git commit -m "$(git log -1 --pretty=%B)"
            git subtree split --prefix target -b gh-pages
            git push -f origin gh-pages:gh-pages

      - uses: actions/cache@v2
        with:
          path: vendor/bundle
          key: ${{ runner.os }}-gems-${{ hashFiles('**/Gemfile.lock') }}
          restore-keys: |
            ${{ runner.os }}-gems-

      - name: Bundle install
        run: |
          bundle config path vendor/bundle
          bundle install

That’s it! We should now have a self-contained automatic workflow for publishing new content to the blog by simply commiting a new Markdown document. We have caching in place to make the process as efficient as possible, and tasks are only executed if something has changed.