ZeroIndexed

Effective terminal presentations with asciinema and doitlive

2022-01-15T00:00:00+00:00

I’ve been trying to push my personal boundaries of “doing open source right”. On the most recent pet project, that means putting those little README shields that people seem so fond of; leveraging GitHub actions for automated testing, linting, and deploying; and recording a screencast to give an interactive tour of functionality.

asciinema is a popular utility for distributing screencasts but the interactive recording style resulted in a final product with quite a few flaws. I wrote a script, built a few helpers, and recorded segments in multiple takes until arriving at something passable. And “passable” just about summarizes the end result, there’s still the occasional typo, the cuts are jarring, and the typing has awkward pauses even with --idle-time-limit=0.3. Worst of all, it’s ~5 minutes long, which I’d reckon is enough to frustrate even the most motivated viewer.

In hopes of doing better, I found a HackerTyper-like tool called doitlive meant for live presentations in the terminal. Using it with asciinema is straight forward (based on this gist by Thomas Applencourt):

asciinema rec --command 'doitlive play --commentecho --quiet --shell bash <script.sh>' <recording.cast>

This runs the asciinema recorder on the outside and typing controls the pace at which doitlive reads the commands from the provided <script.sh>.

Overall the experience is quite pleasant: you write the script and simply whack the keyboard as fast as you please to control the pacing. It also trivializes making future recordings as the behaviour of the underlying demo changes. It’s not quite a non-linear video editor, but it’ll do.

doitlive has features to inject environment variables and display commentary, however the nature of maintaining independent shell state and interpreting commands mean that recording helpers that export variables and change directories cannot be used. There’s also no affordance to execute hidden commands, though that’s understandable given the motivation of a presentation tool.

Here’s the final result, equivalent to the previous one at just 93 seconds. It could be better, and I’m sure there’s someone that prefers the older version, but I’m shipping it and moving on.

Changelog

2022-01-16: Published.

Nix solves the package manager ejection problem

2021-05-30T00:00:00+00:00

This blog post operates on two levels.

On a very direct level, I built a desktop machine with an AMD Radeon RX 5600 XT graphics card and I’ve been fighting a rather annoying interaction with dual monitors on Linux. Here’s how it goes:

After some defined period of inactivity, the system tries to standby / suspend / turn off both monitors.
Both monitors respond to the DPMS commands and briefly blank.
After a few seconds, one of them comes on again, quickly followed by the other one, in an arbitrary order.
XFCE “helpfully” detects this as a monitor hotplug event and pops open a display configuration dialog.

Stepping away to make a coffee now means periodic flickering and coming back to 5 - 10 XFCE monitor configuration dialogs taking over my screen. Very annoying!

Googling around finds this Ubuntu issue from 2018 that seems to describe the same problem, but the documented workaround of amdgpu.dc=0 as a kernel parameter does not work. My graphics card decides to go into a glitchfest and the kernel doesn’t even make it past initrd.

That Ubuntu issue links to this upstream Freedesktop issue, which is old enough to get migrated to this new Freedesktop issue, where some kind soul has put together a patch that applies cleanly on recent kernels. My understanding is that the monitor starts polling its inputs when it enters a DPMS mode, and the patch adjusts the timeouts on the driver to ignore the polling pulses instead of treating them as hotplug events.

So that’s one level: write a blog post, stuff it with enough keywords that other affected users can find it, tell them to USE THE KERNEL PATCH until a fix is integrated into the mainline. A decent public service, but on its own not interesting.

On a second level, I’ve recently been experimenting with NixOS as my daily driver and it rises to this challenge in ways that ArchLinux never could.

Package management on Linux has a colourful history, with various distributions intermixing technology with philosophy to produce whole system configurations that real people can run on real machines with varying amounts of productivity. You have the Slackware “our packages are just tarballs you can extract over /”, the slightly more principled dpkg and RPM’s of the world, the source-first approach preferred ArchLinux, and the “invent the universe to bake a pie” doctrine that is Linux From Scratch.

However, all of them share the fundamental notion of what it means to be a package: a monolithic reusable artifact that can be installed on a user’s machine. (Okay, Linux From Scratch doesn’t really belong in this category, but does it really belong anywhere?)

And there lies the dilemma: if the user needs to go off the beaten path, like patch the damn Linux kernel, they must roll their own package. Sure, some distributions will let you reuse their package building recipe and make the process straightforward, but:

You are now no longer a package consumer, you are now a package developer, which is considered its own distinct category.
Which means you get none of the nice workflows that package consumers enjoy. You are responsible for tracking security disclosures, software updates, and rebasing your changes on top of whatever base you’re building off.

Put another way: the package development workflow does not compose.

In contrast, NixOS makes patching the kernel a four liner:

boot.kernelPatches = [{
  name = "amdgpu-sleep-fix";
  patch = ./amdgpu-sleep-fix.patch;
}];

And the “contrast” here isn’t how easy NixOS makes this (though that is appreciated), it’s that the intention of “build the existing NixOS kernel with just this patch applied” is being purely captured. That’s only possible here because the package management strategy is underpinned by a real programming language where it is derivations all the way down.

There is no distinction between being a “package consumer” and a “package developer” here. I get all the updates that the people responsible for the NixOS kernel put out with no continuous maintenance on my part.

The JavaScript community has a good word for this kind of “stay on the beaten path and get updates” or “do your own thing but carry the maintenance and complexity burden”. They call it ejection.

NixOS (and Nix, which is the practical underpinning that makes this all possible) present a composable solution to this package management ejection problem. And that is interesting!

On Gentoo

An earlier version of this article lumped Gentoo into the same source-first approach as ArchLinux. John Helmert III reached out over email and totony on HackerNews pointed out that Portage has grown to be far more sophisticated than I gave it credit for.

Portage supports a generalized mechanism for patching software that seems like it would have worked just as well here (with fewer lines of code if you will).

I only have a surface level understanding, but it seems like the ebuild itself cannot be patched, so there is no analogue to the Nix override design pattern; meaning there is still some distinction between “package consumer” and “package developer”. However, this is a pragmatic blog post motivated by a real problem, and it seems like Gentoo is quite capable of rising to the specifics of this situation. Fair play!

Changelog

2021-06-06: Credit John Helmert III by name.

2021-05-31: Added a section on Gentoo and Portage.

2021-05-30: Published.

This blog has analytics and a privacy policy now

2021-05-16T00:00:00+00:00

Even though the Jekyll Minima theme has off the shelf support for Google Analytics, I decided to be a good internet denizen and use something that preserved visitor privacy.

I thought that surely in 2021, somebody would have built a simple page view counter that I could just plug into my dinky blog for minimal cost. I ended up evaluating all the popular alternatives: Matamo, Plausible, and Cloudflare analytics.

As it turns out, all of these are rather expensive. What should have been simple became bespoke and Toph was born. It’s a Cloudflare workers service that sits in between the visitors browser and Google Analytics.

Toph is a first party analytics solution, which is generally regarded favourably compared to the third party tracker status quo. It would have been completely okay to do visitor identity tracking by shoving a UUID into a cookie. This usage would likely not even require explicit consent.

But for entirely dubious motivations, I decided to make the implementation cookie-less, trading accuracy and retention metrics for an even smaller privacy footprint. The end result works somewhat like Plausible’s cookie-less tracking, but the Toph implementation is interesting in its own right.

I think if you asked me “how much do you care about web privacy?”, I would have given an apathetic response. But clearly I care enough to spend many hours building a reusable self deployable solution and even packaging it up with Pulumi to make deployment trivial. I’m not sure what virtues this signals.

Oh, and it’s called Toph because it is blind to the visitor’s personal information.

Changelog

2021-05-30: Added this changelog.

2021-05-16: Published.

PGP encryption subkeys are less useful than I thought

2021-05-08T00:00:00+00:00

I recently revisited how I was managing passwords but wanted to maintain the core structure of secrets encrypted via a PGP key. PGP is a good fit here, these secrets are linked to my identity as a person, as opposed to secrets linked to a specific machine or service.

I already have an established PGP key, but this use case demands distributing decryption capabilities to at least my primary computer and mobile phone. My existing PGP key is already used for a hodge podge of purposes, so to limit scope of compromise, it makes sense to generate an encryption and decryption scheme for the specific purpose of securing my passwords.

I thought I could simply generate a new PGP subkey tied to my main identity and this idea seems to have some colloquial support:

Use one primary key for each identity you need, otherwise, use subkeys.

[…]

Examples for using subkeys:

You want to use multiple keys for multiple devices (so you won’t have to revoke your computer’s key if you lose your mobile)

tl;dr: Don’t bother. The PGP ecosystem does not support encryption subkeys for a specific purpose. You should just generate an entirely separate PGP key for your use case. For me securing passwords, I’m not even bothering to publish this new key to keyservers. The rest of this blog post is dedicated to describing the various rabbit holes I explored before settling on this “don’t bother” conclusion.

PGP identities and keys

The previous paragraphs have used the words “identity” and “key” rather loosely, so let’s bring some rigour here (though I’m optimizing for clarity and not pedantry so caveat emptor).

At the top level for PGP, there is a private/public keypair called the mainkey. This keypair is used to certify various subkeys as belonging to the mainkey.

Each subkey has associated key flags which determine what operations this subkey is valid for. Besides ontological clarity, this distinction contributes to the security properties of the overall system:

If you look into the details of the math of public-key encryption, you will discover that signing and decrypting are actually identical operations. Thus in a naïve implementation it is possible to trick somebody into decrypting a message by asking them to sign it.

The mainkey is also used to certify various user ids (also known as UID, identity), which usually ties the mainkey to one or more real world names and email addresses.

For more on this topic, the clearest explanation I can find is Dave Steele’s anatomy of a GPG key.

The ecosystem is simplistic

The initial thought is to simply associate some metadata with a specially generated subkey that identifies it as special-only-for-explicit-personal-use. The vague hope is to prevent others from implicitly using this key when sending me encrypted email or sharing work related secrets.

Unfortunately, the ecosystem doesn’t support any metadata of the sort. You can force GPG to use a specific subkey with an explicit invocation:

When using gpg an exclamation mark (!) may be appended to force using the specified primary or secondary key and not to try and calculate which primary or secondary key to use.

However the default behaviour for GPG makes it easy to do the wrong thing:

If you have multiple encryption subkeys, gpg is said to encrypt only for the most recent encryption subkey and not for all known and not revoked encryption subkeys.

In a somewhat similar vein, OpenKeychain will encrypt to all subkeys with the encryption flag set, regardless of metadata:

Since v 3.4 we encrypt to all subkeys with the capability flag for encryption: 7648602

Overall, this means that ecosystem support for any sort of metadata of encryption subkey purpose is weak. It cannot be relied upon without rolling our own tools, which is already more work than I’m willing to get into.

Key usage flags are not sufficient

I’m not the first person to have thought of using subkeys in this fashion, here’s a mailing list post by Mike Cardwell from almost a decade ago that’s short enough to quote in its entirety:

If I have more than one signing subkey in my keypair, is there a way of advertising the purpose of each subkey with the public key that people download? Eg:
This subkey is for signing email only
This subkey is for signing sourcecode only
I’ve considered generating an entirely separate keypair and then cross-signing them, but that seems inelegant.

To which there is a helpful reply by Daniel Kahn Gillmor:

My first thought was to look up the list of standardized “key usage flags”, which are defined here: https://tools.ietf.org/html/rfc4880#section-5.2.3.21

Looking at the quoted RFC, there are no “key usage flags” that explicitly cover the usage I intend with my password management subkey:

0x01 - This key may be used to certify other keys.

Almost certainly not, the password subkey should not be certifying other keys.

0x02 - This key may be used to sign data.

No, we don’t care for signing, just encrypting.

0x04 - This key may be used to encrypt communications.

For the password use case, this flag ought to be true. But this alone is ambiguous because there are other keys, like encrypted email, that would also have this bit set.

0x08 - This key may be used to encrypt storage.

This is the most promising flag but the RFC does not standardize the distinction:

Note however, that it is a thorny issue to determine what is “communications” and what is “storage”. This decision is left wholly up to the implementation; the authors of this document do not claim any special wisdom on the issue and realize that accepted opinion may change.

In the 14 years since the publication of that RFC, it seems like implementations have not evolved a conventional distinction either. Here’s the GPG source code:

      /* We do not distinguish between encrypting communications and
         encrypting storage. */
      if (flags & (0x04 | 0x08))
        {
          key_usage |= PUBKEY_USAGE_ENC;
          flags &= ~(0x04 | 0x08);
        }

Which only leaves us with the last defined key usage bits:

0x10 - The private component of this key may have been split
       by a secret-sharing mechanism.

0x20 - This key may be used for authentication.

0x80 - The private component of this key may be in the
       possession of more than one person.

None of these are relevant to the encryption-for-passwords use case. Passwords are used for authentication and 0x20 mentions authentication, but the intent is for PGP subkeys that directly participate in the authentication scheme — like a PGP subkey that can also be an SSH key — instead of the indirect workflow of “subkey decrypts password then actor uses password to authenticate”.

Subkey notations have no adoption

Going back to Daniel Kahn Gillmor’s mailing list response, they suggest an alternative form of subkey metadata called “notations”:

Instead, you could add a notation to the subkey signatures. https://tools.ietf.org/html/rfc4880#section-5.2.3.16

[…]

You might want to discuss it with the (very low-traffic at the moment) working group: IETF OpenPGP Working Group <ietf-openpgp at imc.org>

To summarize the RFC: notations are a standardized mechanism to add arbitrary name/value octet sequences to a subkey. The RFC even mitigates name collisions, private use should have an @ while standardized notation names have no @ but need to be registered with IANA.

Even though the RFC expends the energy to standardize notations, implementations and conventions haven’t really capitalized on this affordance. This StackExchange answer notes:

Notations are rarely used in practice (actually I’ve never seen real usage, although there are few notation subpackets to be found in key server dumps). Example usage might be to add the location of an identity check during key signing, or the document types presented.

As far as I can tell, there are no actual standardized notation names. The IANA registry for PGP notations is empty.

I did find this IETF internet draft that expands RFC 4880 to specify some standardized notation names and intended usage, but:

None of the 10 new notation names would fit this purpose of marking an encryption key as not fit for general communications.
The internet draft has expired and I don’t know if it’ll be picked up again and driven to consensus. Although it has been through 10 revisions and the expiry was only ~5 weeks ago, so clearly some people care.

User IDs are not linked to subkeys

The final idea is to put this metadata in the comment field of a user ID linked to the mainkey. I’m guessing that this mailing list post is after similar goals (though it’s phrased in classic XY fashion, so who knows):

Is it possible to have subkeys with different comments than the main key? How?

The mailing list question is based on a misunderstanding, user IDs can have arbitrary comments but they are not linked to subkeys in any way:

Both subkeys and user IDs are related to a mainkey. In this sense user IDs and subkeys are on the same level. There is no such thing as a subkey user ID or a user ID subkey. User IDs are just “names” for a mainkey. You can add and remove user IDs and subkeys. They do not affect the other group.

So besides being unstandardized, user ID comments are simply the wrong tool for this purpose.

Takeaways

Just generate a separate PGP mainkey for this specific purpose. Give it a distinct user ID for quick identification, I used a plus sign on Gmail to mark the email address. Consider if this mainkey even needs to be published on public keyservers, I did not bother since this key is for private use and does not represent a person.

In May 2021, trying to use subkeys would be actively harmful. If the subkey were published alongside your existing mainkey, implementations would implicitly use it to encrypt communications intended for you. This adds additional awkwardness to the poor UX that PGP email already suffers.

Changelog

2021-10-30: Dave Steele reached out over email and pointed out that GPG can indeed encrypt to an explicit subkey with an exclamation point syntax. The earlier reference that claimed this couldn’t be done has been removed.

2021-05-09: Published.

A blog begins…

2021-04-18T00:00:00+00:00

I’ve been putting off setting up a blog for far too long. There’s been a bunch of content worth talking about, but it’s been blocked on this grandiose ambition to rethink the medium of the digital written word. If you wish to write a blog from scratch, you must first invent a whole new blogging engine.

Hopefully that’ll still happen. For now, let’s try low fidelity things and validate that I can even commit to the blogging thing. GitHub Pages. Jekyll. Minima. All content, zero yaks to shave.

Besides the usual first blog post drivel, this post accomplishes two additional things:

It demonstrates that Jekyll is properly configured to extract excerpts and put that content on the home page.
It forms the starting point of a personal challenge. It’s 2021-04-18, let’s see how long and how many system transitions the “permalink” of https://zeroindexed.com/jekyll-begins can survive.

Changelog

2021-05-08: Added this changelog.

2021-04-18: Published.