Graham F. Scott: Blog

The books I read in 2025

2026-01-01T18:00:00Z

It wasn't my intent, but quite a few of these turned out to be about work, labour, and power. In every sense of the word, folks were on the hustle, either head-on as in The Hammer, Offshore, and The Soul of a New Machine, or more obliquely, as in Niko Stratis's The Dad Rock That Made Me a Woman or Colson Whitehead's Crook Manifesto.

Ancillary Mercy, by Ann Leckie
Ancillary Sword, by Ann Leckie
Audition, by Katie Kitamura
Crook Manifesto, by Colson Whitehead
The Dad Rock That Made Me a Woman, by Niko Stratis
Everything Must Go: The Stories We Tell About the End of the World, by Dorian Lynskey
Fellow Travelers, by Thomas Mallon
The Hammer: Power, Inequality, and the Struggle for the Soul of Labor, by Hamilton Nolan
The Library: A Fragile History, by Andrew Pettegree and Arthur der Weduwen
Offshore: Stealth Wealth and the New Colonialism, by Brooke Harrington
Pick a Colour, by Souvankham Thammavongsa
Proto: How One Ancient Language Went Global, By Laura Spinney
The Quakers: A Very Short Introduction, by Pink Dandelion
The Soul of a New Machine, by Tracy Kidder
The Theory and Craft of Digital Preservation, by Trevor Owens
Way Down Deep in the Belly of the Beast: A Memoir of the Seventies, by Douglas Fetherling
When Women Ran Fifth Avenue: Glamour and Power at the Dawn of American Fashion, by Julie Satow
Will There Ever Be Another You, by Patricia Lockwood

Notes

In Way Down Deep..., Douglas Fetherling calls Toronto of the 1970s “the world’s most cosmopolitan and least sophisticated city” and unfortunately it still feels witheringly accurate today.
The Hammer and Offshore make for a bracing, though somewhat depressing, pairing. The decline of unionized labour and the rise of the (latest) plutocrat class are obviously linked in the abstract, but both of these books zoom in to the human-level dynamics — how people actually organize their workplaces now (or fail to), and how wealth managers actually conceal the tidal flows of capital that slosh around the globe from one tax haven to the next. They’re also both satisfyingly lean, cutting to the chase to make their arguments and not outstaying their welcome.
Brooke Harrington also makes a point about “New Colonialism” in Offshore that seems obvious once she points it out: it’s not a coincidence that so many of these tiny island tax havens around the world are former British colonies. While the UK’s formal colonial structure crumbled or withdrew from these far-flung holdings, many imperial institutions seamlessly continued their function of covertly channeling the world’s wealth, largely still to the benefit of London, but now in service of the City instead of the Crown.
The Dad Rock That Made Me a Woman is a powerful mix of memoir and criticism, but the essay that’s really stuck with me is the one about Bruce Springsteen’s “Dancing in the Dark.” Positioned mid-point in the book, it describes a moment of extreme emotional darkness that becomes the turning point toward something better, encapsulated by Bruce’s snarl into the mirror: “wanna change my clothes, my hair, my face.” I’d never really heard the depth of anguish packed into that line, or its corresponding potential for transformational change. Having read this book, I'll never hear that song the same way again (complimentary).

Good boots: Salomon X-Ultra 5 Trail Shoes

2025-12-29T18:00:00Z

After more than 10 years, it’s finally the end of the trail for my Salomon X-Ultra 5 hiking boots. They go to their reward having safely delivered me across the deserts of Utah, the mountains of the Yukon, and the mossy cliffs of the Faroe Islands, among many other places.

They were good boots because they were both sturdy and lightweight, and provided a good balance between support and flexibility. Almost every time I wear my mid-rise equivalents, which come up past the ankle, I wish I was wearing these instead. However, the tread has worn off in places and there are holes at several flex points now, so it’s time.

I was wary at first of the Quicklace™ system, where the lace is a closed loop that you tighten with a cleat. After all, if the proprietary shoelace breaks out in the middle of nowhere, that could be a long hobble back, and replacement would be more involved. However, I’ve come around because I’ve never had such a problem in a decade of steady wear, and it’s so much faster and more convenient than traditional shoelaces that I think it’s worth the tradeoff.

I intend to replace them with the contemporary equivalent, and I hope to cover just as much ground in them.

The boots, pictured in Iceland, Scotland, and the United States.

Eleventy pagination with Vento

2025-08-29T00:00:00Z

I recently switched my Eleventy site from Nunjucks to Vento for templating. Mostly I just wanted to try it out, but I find I do prefer the {{ }} syntax over {% %}, and the ability to run vanilla JavaScript inside tags is nice.

I won't rehash Chris Kirk Nielsen's thorough post on migrating to Vento, which came in very handy. But I'll share what I learned the hard way about Eleventy pagination in Vento.

Eleventy documents a good example of how to handle pagination with Nunjucks. Following that example, my original pagination component looked like this:

{% if (pagination.pages.length > 1) %}
<nav class="pagination">
  <span>Page: </span>
  {% for pageEntry in pagination.pages %}
    {% if page.url == pagination.hrefs[ loop.index0 ] %}
      <span class="pg" aria-current="page">{{ loop.index }}</span>
    {% else %}
      <a href="{{ pagination.hrefs[ loop.index0 ] }}">{{ loop.index }}</a>
    {% endif %}
  {% endfor %}
</nav>
{% endif %}

Migrating this component was the one place where I ran into real trouble, and the reason is that the documented method relies on “special variables” that are peculiar to Nunjucks. That magic loop.index0 value — which represents the current zero-indexed iteration of the for loop — is a nifty shortcut, but it doesn’t exist outside Nunjucks. You can easily swap the syntax but, of course, those special variables no longer mean anything.

After some blundering around, I figured out that Vento can in fact give you the index of a for loop (although the docs don’t exactly articulate it that way). I suspect others might run into similar issues and find this useful.

Here's the equivalent Vento pagination component that I eventually landed on:

{{ if (pagination.pages.length > 1) }}
<nav class="pagination">
  <span>Page: </span>
  {{# `pagination.pages` is an array of arrays, so `index` here acts as a counter #}}
  {{ for index, _content of pagination.pages }}
    {{if pagination.hrefs[index] == page.url}}
      <span class="pg" aria-current="page">{{index + 1}}</span>
    {{else}}
      <a href="{{pagination.hrefs[index]}}">{{index + 1}}</a>
    {{/if}}
  {{/for}}
</nav>
{{ /if }}

Note that _content isn't used at all, but if you don't declare it, the array isn't destructured and everything breaks.

The books I read in 2024

2025-01-01T14:00:00Z

Only 13 total this year, although one of them was a biggie.

The Age of Innocence, by Edith Wharton
American Midnight: The Great War, a Violent Peace, and Democracy’s Forgotten Crisis, by Adam Hochschild
Everything and More: A Compact History of ∞, by David Foster Wallace
How Infrastructure Works: Inside the Systems That Shape Our World, by Deb Chachra
Hyperobjects: Philosophy and Ecology after the End of the World, by Timothy Morton
The Late Americans, by Brandon Taylor
The Maniac, by Benjamín Labatut
Middlemarch, by George Eliot
Parable of the Sower, by Octavia E. Butler
Parable of the Talents, by Octavia E. Butler
Super-Infinite: The Transformations of John Donne, by Katherine Rundell
Warez: The Infrastructure and Aesthetics of Piracy, by Martin Paul Eve
The Wounded World: W. E. B. Du Bois and the First World War, by Chad L. Williams

Notes

Middlemarch was the major project for this year and it was a delightful surprise. I went in knowing basically nothing about it and it took some time to acclimate to the Victorian prose, but it's beautifully funny, wise, and humane. I admit I glazed over at some of the political specifics, but the social dynamics are so finely observed that they’re perfectly legible to a modern reader.
Warez was a learning experience in more ways than one, since I read it in the process of converting it from PDF to EPUB format (ironically, for a book about piracy, with the permission of the author).
An unexpected theme of American Midnight, but one that resonated this year, was how much the New York Times sucks, and always has. Whether it's peddling braindead jingoism in the leadup to the USA’s entry into the First World War, laundering hysterical Red-baiting in its aftermath, or simply being an easy mark for manipulative ghouls like J. Edgar Hoover, the Times has a long and inglorious history of witless bootlicking.
Originally published in 1993 and structured as a series of diary entries, Parable of the Sower begins on July 20, 2024 — coincidentally around when I picked it up. Reading it today, its grim predictions about climate collapse, social breakdown, and political dysfunction feel eerily on the nose.
Finally picked up and finished two books that I’d stalled out on in prior years: Everything and More, and Hyperobjects. Both dense and demanding, but ultimately rewarding, and with some surprising intersections.

100 stars

2024-12-08T19:11:55Z

This week, 1,779 days after publishing the first version of the first plugin, Embed Everything earned its 100th GitHub star. It’s a nice little milestone, and a good opportunity to take stock.

By the numbers

A snapshot of key stats about the project as of today:

1,512 commits
20,455 unit tests
1,924,929 npm downloads^[1]

More important than the vanity metrics of GitHub stars or npm downloads is the dependents graph, which, to me, provides a better picture of how the plugin is actually used in the wild. There are 572 public repos using it, and 6 packages that include it as a sub-dependency.

I've spent quite a lot of time scrolling through the dependents list, snooping into people’s codebases to see how they use this tool. One thing I notice, and that I believe is a good sign, is that many of them add no configuration at all. To me, that's a positive signal that I’ve chosen the right defaults, and that the plugin fits users’ mental models — it behaves pretty much the way they expect it to.

It’s likely there’s a sizeable contingent of users who don’t know the plugin exists at all. It’s been bundled into a few 11ty starter templates such as Pack11ty, Elva, and Spacebook, so for some people the core functionality Just Works™ and they neither know nor care how it happens. I find that oddly gratifying! It was the whole point, after all — to remove a few needless mouse-clicks, so you can stay focused on the task at hand. It should be invisible.

The future

At this point the plugin is pretty mature; the pace of development is naturally going to be slower, and I think that’s fine. Still, there are things to do.

End-to-end testing

I added some end-to-end testing with Playwright a few years back but then removed it because my implementation was kind of flaky. The plan was always to add it back in, and I still intend to do that.

Vitest

Still thinking about this one. I’ve worked with Vitest recently and I like it. In particular, I think it makes sense because, compared to Ava, it includes better built-in support for mocking, and that’s important for testing oEmbed implementations. But it would be quite the chore to switch. We’ll see!

ESM

I’m in no rush on this one, but eventually. Now that Node.js 22.12.0 has turned on require(esm) by default, I think we're in the final throes of the CJS/ESM wars. Overall, if tradeoffs arise between developer ergonomics for me versus stability for end users, I intend to choose the latter. I expect the codebase to remain CJS for at least a couple years yet.

Preprocessor implementation

An intriguing feature added in Eleventy 3.0 was Preprocessors, which allows you to manipulate markdown content earlier in the data pipeline. I've got a little test implementation working and I think it's promising. Among other things, it means not having to parse HTML with RegEx, and the ability to override the config on a per-post basis using just the frontmatter. This looks to me like the most logical path to a 2.0. More to come.

This is the total for all the plugins combined, so there's some double counting, since the everything package downloads all the others. ↩︎

Archiving your Twitter account with Eleventy

2024-11-20T23:30:00Z

Last week, I finally deleted all my tweets^[1]. I haven't used Twitter for some time, and hadn’t posted in years, but the latest round of vaguely sinister terms-of-service updates prompted me to pull the plug for good.

However, I wanted to keep an archive of my account, on my own terms. I wanted it to be lo-fi, low-maintenance, and low-cost. I'm happy to report that it worked: I’ve now got a complete record at twitter.gfscott.com. Here's how I did it.

How it works

The site is an Eleventy project, which uses the tweets.js file that's part of your Twitter archive download. Basically it just churns through that file, generating a permalink for each tweet. The final product is just a giant pile of static HTML files.

I've deployed my personal archive to Cloudflare Pages, so it costs me nothing and could easily be moved if needed.

Create your own

If you'd like to make your own version, you can use this GitHub template. All the instructions are in the readme:

Download your Twitter archive.
On the GitHub template repo, click Use this template, then select Create a new repository. Fill in the required details to create your own repo.
Copy the contents of {archive}/data/tweets.js into ./src/_data/__tweets.js.
Change window.YTD.tweets.part0 = to export default.
Copy image files from {archive}/data/tweets_media/ into ./src/static/images/twitter/.
Run npm run dev to preview.

Caveats and limitations

My aging desktop outputs about 1,000 tweets/sec, so expect build-times to scale with how prolific you were.
The conversational context is incomplete. You can see who you were replying to, but not what they said. So it's a one-sided conversation.
No threads. Given the available data, I don't see a way to group posts together, but perhaps others have ideas. Happy to discuss in an issue or PR.
The site replaces t.co links with the actual URL when possible, but (especially in the early days of Twitter) there were a lot of link shortener services that have since vanished, leaving irretrievably broken links behind. Sad!
I don't think there's any truly sensitive data in the tweets.js file you get in the archive download, but I'm keeping mine in a private repo all the same.

Alternatives

Marco Maroni has created a Twitter-to-Bluesky tool, which imports and backdates tweets into your Bluesky account. I decided that I wanted a clean break, so I haven't tried it out, but others have and it seems pretty cool.

I used this script, which worked well and was free. I’m keeping the username, just to prevent it from falling into enemy hands. ↩︎

Save the bike lanes

2024-10-27T18:00:00Z

The Environmental Registry of Ontario asked for comment on the province’s plot to kill Toronto’s bike lanes. Comments are open until November 20, 2024.

Here's what my submission said:

As someone who actually lives in a high-density urban environment, I can say from first-hand experience that adding bike lane infrastructure has made my life better and safer. I'm a frequent pedestrian, occasional cyclist, and occasional driver. In all these situations, I appreciate the presence of separated bike lanes. Even if bike lanes slowed city traffic — which they don't — I would still be in favor.
This government should not backpedal on the significant improvements that have been made to Toronto’s street infrastructure in the last few years. The obvious underlying instinct at work with this proposal — removing existing bike lanes to appease a small number of frustrated drivers who don't actually live here — shouldn't be the basis for public policy. A few people’s road rage doesn't make for good law.
If Queen’s Park insists on tinkering with municipal affairs to improve the city's traffic situation, here are some alternative policies:
Automated speed cameras, to enforce the speed limit (something the police^[1] have evidently abandoned)
Automated red-light cameras, to penalize reckless driving that endangers pedestrians, cyclists, and other drivers
Automated Block-the-Box cameras, to discourage drivers from blocking cross-traffic in intersections (another neglected police duty)
Enforcing no-stopping zones (yet more police inaction), so the whole street doesn't grind to a halt for a single Uber or DoorDash driver with their four-way flashers on.
Expanding the no-parking hours on major thoroughfares. There are large segments of the day where Avenue Road, Mount Pleasant Road, Bathurst St, Lawrence Ave, Eglinton Ave, and more effectively lose the use of entire lanes in both directions because of a handful of (legally) parked cars.
More frequent and more reliable GO Train service, to make rail commuting viable for more users, from greater distances
A downtown congestion fee to incentivize more efficient use of city streets, and to pay for the other required improvements
The behaviour of drivers, not cyclists, has created Toronto’s traffic problems; only changes to driver behaviour can fix them. I suggest you start there.

Now, do I find it extremely galling to be writing crank letters to the government? Yes I do. Do I think it’s ultimately a futile gesture? Also yes. But it's a stupid policy, even by the degraded standards of this government, and they deserve to hear it.

There are very few problems to which I belive the solution is “more cops.” But honestly, traffic enforcement is one of them. ↩︎

The books I read in 2023

2024-01-01T17:00:00Z

This year I focused on completing some book series, while tracing back the original influences for others. Overall number is down this year, so something to work on.

Ada, the Enchantress of Numbers, by Betty Alexandra Toole
The Dark Forest, by Liu Cixin
Death’s End, by Liu Cixin
Dinosaurs, by Lydia Millet
Eileen, by Otessa Moshfegh
In Lonely Places: Film Noir Beyond the City, by Imogen Sara Smith
La Belle Sauvage, by Philip Pullman
Maurice, by E. M. Forster
Midnight in the Garden of Good and Evil, by John Berendt
A Paradise Built in Hell: The Extraordinary Communities That Arise in Disaster, by Rebecca Solnit
Paradise Lost, by John Milton
The Secret Commonwealth, by Philip Pullman
Stay True, by Hua Hsu

Notes

After reading The Three Body Problem last year, I kept going with Liu Cixin’s follow-ups, The Dark Forest and Death’s End. I appreciated the ambition of the narrative, which spans millennia and sprawls across the cosmos, but the human-scale characterization suffers as a result, and in the end I found its worldview to be too bleak and misanthropic.
In contrast, it felt great to slip back into Philip Pullman’s world with La Belle Sauvage and The Secret Commonwealth, which expand on the original His Dark Materials trilogy (with a third on the way). He smuggles in his political agenda more artfully than Liu does, and it’s a more humane, generous vision overall, disguised as a swift adventure yarn.
I followed the thread back from Pullman to his source, Paradise Lost, and man, what a banger. I see why it was such a splash at the time, and why it’s endured for centuries.
I’ve been searching for a proper, thorough biography of Ada Lovelace, and it seems like a curious blank spot. Ada, the Enchantress of Numbers comes close, but it’s an odd specimen, more a series of biographical sketches sprinkled with excerpts from letters and other assorted commentary. It’s good, but still not quite the thing I’m looking for. The search continues.
I’ll say the same thing about A Paradise Built in Hell that I do about many books, which is that it's very good, and would be even better at a third the length. Most non-fiction books are too long! The book is sort of a mirror image of Naomi Klein’s Shock Doctrine, where crisis again acts as a catalyst for change — but in this case a positive one, where disasters allow people to glimpse the utopian potential for collective action, mutual aid, and community solidarity. Which is perfectly interesting — but the thesis is not bolstered by many, many extra examples illustrating this same point over a couple hundred pages.

Good song: “How We Drift Away” by Tim Heidecker and Weyes Blood

2023-11-07T21:00:00Z

The first reason I like this song is its ’70s throwback vibe. That melodic bassline, the steel pedal guitar wailing in the middle distance, Natalie Mering’s Quaalude-deadpan delivery. Big-time Carpenters vibes. Here for it.

The second reason is that it explores comparatively uncommon emotional terrain for pop music: the vague melancholy of an eroding platonic friendship. I think most adults can relate to the sensation of meeting up with someone you once felt close to and finding the connection threadbare. The lyric You can take me out / of your phone describes, with deadly specificity and concision, a hypermodern social dynamic. Deleting someone from your phone is trivial, technically. But socially, it’s fraught — the final act of deliberately pulling the plug on a relationship, no matter how feeble its life-signs might have been. That shit is real.

The third and final reason I like this song is the wild narrative turn it takes in the third verse, abruptly jumping 20,000 years into the past: In the Chauvet caves / women painting on the wall. I love the grandiosity of this gesture, grasping at common human experiences across the centuries. It reminds me of Terence Malick’s The Tree of Life, whose kitchen-sink family drama plays out against a backdrop of the literal creation of the universe and the end of the world. By juxtaposing the intimate and the infinite, we glimpse how they’re part of the same whole. Our individual lives are so vanishingly small, and yet, it’s all we have; it’s everything. Some works of art deliver penetrating observations of human nature. Some grapple with cosmic mystery. Few manage to do both at once. To me, this is one of them.

How to merge multiple Git repos and keep their commit history

2023-03-18T19:00:00Z

Background

I maintain an Eleventy plugin, Embed Everything, which aggregates a bunch of other plugins. Until recently, I was keeping a separate Git repository for each package. It was getting to be a hassle, having to track issues, dependencies, and maintenance tasks across nine different repos. So after some experimentation, I decided to combine them all into a single monorepo. Here's how I did it.

This process is based largely on this blog post by Willem Cheizoo. I very slightly modified Willem's process, however, so I wanted to document it here.

Goals

Merge several separate repos into one.
Preserve the complete Git commit history. This was important because I wanted to ensure that volunteers continue to receive credit for their contributions to the codebase.

# Before: two repos
├── everything
└── youtube

# After: two packages, one repo
└── monorepo
    └── packages
        ├── everything
        └── youtube

Overview

This is an overview of the basic steps. I'll go through each one in greater detail.

Re-arrange each repo so they can merge without conflicts
Add each repo as a remote source for the monorepo
Merge the repos with --allow-unrelated-histories

Prepare each repo for merging

The first task is to get each repo into the target file structure. It's important to do this ahead of time, to prevent merge conflicts.

Checkout a working branch, so we can bail out safely:
$ git checkout -b monorepo-prep
Create the target folder structure:
$ mkdir packages && mkdir packages/everything
Move files into that new structure. I like using git mv:
$ git mv package.json packages/everything
Repeat this step as necessary, until all the files are moved.

# Before: 
└── everything
    ├── README.md
    ├── index.js
    ├── package.json
    └── ...

# After: 
└── everything 
    └── packages
        └── everything
            ├── README.md
            ├── index.js
            ├── package.json
            └── ...

Finally, commit all your changes:
$ git add -A
$ git commit -m "Prep for monorepo migration"

Repeat for each repo

Follow this same process for each repo that you're planning to merge. The goal is to ensure that they all have parallel file structures. If they don't, you're likely to get git conflicts that are a pain to reconcile.

In my case, I did the same thing for the youtube repo:

# Before: 
└── youtube
    ├── README.md
    ├── index.js
    └── package.json

# After: 
└── youtube 
    └── packages
        └── youtube
            ├── README.md
            ├── index.js
            └── package.json

We're now ready to merge the two repos into one.

Connect the repos

We'll allow these separate repos to communicate by connecting them using git remote.

Typically, you'd use the git remote command to connect your local repo to an upstream repo over the network, such as GitHub. But you can use it to track any repo, including the ones on your local file system:

# ./everything
# -f          = <f>etch the list of branches from the remote repo
# youtube     = <name> for the remote repo
# ../youtube  = <url> of the "remote" repo. In this case, just a relative path

$ git remote add -f youtube ../youtube

So the everything repo is now tracking the youtube repo as a remote source. You're now ready to merge these separate repos into one.

Merge the repos

We'll use git merge to pull the youtube repo into the everything repo, including its complete commit history. The --allow-unrelated-histories flag is what makes this whole thing work.

By default, a successful merge command creates a commit. I like using the --no-commit flag so I can confirm the new file structure before committing the changes, but it's optional.

# ./everything
# youtube/monorepo-prep       = <remote repo name>/<branch name>
# --allow-unrelated-histories = It's OK to smoosh these repos together
# --no-commit                 = (Optional) Stage instead of committing

$ git merge youtube/monorepo-prep --allow-unrelated-histories --no-commit

A this point you can validate that the files were moved as expected:

└── everything
    └── packages
        ├── everything+       └── youtube

You can run git log --all to check that the commit history was merged. (The --all flag lets you view the merged commit history before completing the commit.) The two histories are merged together in reverse chronological order.

If you're satisfied that everything worked as expected, you can git commit the results. At this point, you've successfully merged the two repos, including their complete Git commit history.

Cleanup

There are several things you might want to do at this point. There are some files in each package directory that are likely no longer needed. For example, you can keep a single .gitignore file in the project root and delete all the individual ones in each package. You can also git remote rm youtube, since you probably don't need that remote connection anymore. And there are other tasks required to set everything up as a monorepo, but that's a different post.

Speedrun

Here's a condensed run-through of this full procedure as it looks on the command line:

# ./
$ cd everything/
$ git checkout -b monorepo-prep
$ mkdir packages
$ mkdir packages/everything
$ git mv index.js packages/everything
  # ... repeat the above step for all files
$ git add -A
$ git commit -m "Prepping monorepo"
# switch to the youtube repo
$ cd ../youtube
$ git checkout -b monorepo-prep
$ mkdir packages
$ mkdir packages/everything
$ git mv index.js packages/youtube
  # ... repeat the above step for all files
$ git add -A
$ git commit -m "Prepping monorepo"
# switch back to the monorepo folder
$ cd ../everything
$ git remote add -f youtube ../youtube
$ git merge youtube/monorepo-prep --no-commit --allow-unrelated-histories
# Inspect that everything worked as expected. If so:
$ git add -A
$ git commit -m "Merging repos"
$ git remote rm youtube
# Done!

The books I read in 2022

2023-01-01T17:00:00Z

I leaned toward novels this year, while belatedly picking up some older non-fiction titles that had been on my to-read list for some time.

Archives, by Andrew Lison, Marcell Mars, Tomislav Medak, and Rick Prelinger
The Boy Kings: A Journey Into the Heart of the Social Network, by Katherine Losse
Cleanness, by Garth Greenwell
Dark Age Ahead, by Jane Jacobs
The Heat of the Moment: A Firefighter’s Stories of Life and Death Decisions, by Sabrina Cohen-Hatton
The Free World: Art and Thought in the Cold War, by Louis Menand
Harlem Shuffle, by Colson Whitehead
Lapvona, by Ottessa Moshfegh
The Listeners, by Jordan Tannahill
Paper Girls, By Brian K. Vaughan and Cliff Chiang
Pure Colour, by Sheila Heti
The Real World of Technology, by Ursula M. Franklin
Sea State, by Tabitha Lasley
Second Foundation, by Isaac Asimov
A Separation, by Katie Kitamura
Son of Elsewhere: A Memoir in Pieces, by Elamin Abdelmahmoud
The Three-Body Problem, by Liu Cixin
The Topeka School, by Ben Lerner
The Trouble with Brunch: Work, Class, and the Pursuit of Leisure, by Shawn Micallef

Notes

I picked up The Free World on a whim and while I had expected it to be about “the CIA invented abstract expressionism!” and so on, it’s more of a series of potted biographies of key midcentury artists and academics; the Cold War is simply a backdrop to it all (which is fine!). Very breezy and engaging.
Feeling so annoyed with humanity that you deliberately provoke alien invasion? Though it was originally written in 2008, The Three-Body Problem fit my 2022 mood quite well.
After reading Seeing Like a State last year and finding it interesting but ponderous, Ursula Franklin’s 1989 Massey Lecture series, The Real World of Technology, made for an illuminating contrast. It’s lucid and concise, with a similarly sweeping view of history but a more thoughtful moral argument at its core. It shows its age in parts, but is otherwise uncannily relevant to the technological discourse of 2022.
I finished Asimov’s Foundation trilogy and I think I'm going to leave it at that.
Mixing memoir and reportage is not a new trick, but Tabitha Lasley turns the dial to 11 with Sea State. Both an account of the brutal labour conditions of North sea offshore oil drilling, and a vivid first-hand encounter with the equally extreme emotional terrain shaped by those conditions. Thrilling and cringe-inducing in the sheer recklessness of the project, with the narrative arc of a novel but the frank prose of a reporter whose beat is her own unraveling life. Strange and compelling.

Denyse Thomasos at the AGO

2022-11-25T18:00:00Z

“Metropolis”, 2007, by Denyse Thomasos. Collection of the Art Gallery of Ontario.

This week I had a day off and decided to go to the Art Gallery of Ontario, with no particular agenda. And I was absolutely blown away by the exhibit Denyse Thomasos: Just Beyond.

I’d never heard of Thomasos (1964–2012) before, or seen any of her work. That added to the thrill of wandering aimlessly up to the fifth-floor gallery and encountering this art for the first time, without any preconceived ideas.

What I like most about these paintings is the way they skate the line between abstract and figurative, organic and manufactured. Big, drippy expressionist strokes, interlaid with regimented, architectural geometries.

Bone structures, shipping containers, urban infrastructure, prison cells — you get these snatches of recognition without ever quite fixing your eye on a recognizable object. Dense layering means there’s always something pulling your gaze elsewhere on the canvas. They're easy to get lost in.

Two paintings by Denyse Thomasos. Left: “Dos Amigos (Slave Boat)”, 1993, Collection of Cadillac Fairview; Right: “Virtual Incarceration”, 1999, Estate of Denyse Thomasos and Olga Korper Gallery.

The political content here is pointed, without ever being didactic. One painting, Virtual Incarceration (11 feet tall by 20 feet wide) is monumentally large, but still manages to produce a cramped and crowded visual effect, with warped and unsettled perspective planes. I find it rare for abstract painting to achieve this kind of synthesis, where the visual and conceptual content complement one another so clearly.

Anyway, I really liked this show! It’s delightful to stumble over something you’ve never seen before and feel so instantly at home with it. Just Beyond is at the AGO through February 2023.

The Age of Participation

2022-06-19T17:00:00Z

Front and back views of a volunteer T-shirt from ChangeCampTO, February 2010.

Recently I was clearing out some old stuff that no longer sparked joy™ and decided the time had come for this T-shirt. Instead of thanking the shirt, however, I thought it was more meaningful to eulogize the era for which it’s personally emblematic.

I got it in exchange for being a volunteer facilitator — a “change agent” — at ChangeCampTO in 2010. ChangeCamp was an “unconference” — a term that dates it to an extremely specific moment in our shared socio-digital fossil record. Here’s how the organizers described it:

ChangeCamp is an event format, an open community and a set of tools and ideas designed to give citizens and governments the ability to work collaboratively in new ways to make change and to better address real-world challenges in our communities.

The event itself was held at the Metro Toronto Reference Library on February 16, 2010. Since it was an unconference, there was no firm agenda beyond a “design exercise” in which participants, in groups of 8, considered questions like “How do we re-imagine government and citizenship in the age of participation?” As facilitator for one group, my job was to try to keep my table’s conversation on track, as well as documenting it in real-time through a live-blog format (We used ScribbleLive). I don’t remember any specifics of that conversation today, but I remember the vibe with perfect clarity.

These ideas were everywhere in the late ’00s and early ’10s. I’d summarize the ethos this way:

The Internet is, broadly speaking, a democratizing force;
Open Source, which contributed so much to the creation of the Internet, is a viable model for broader sectors of society;
Transparency and openness naturally foster more humane governance;
Innovation and efficiency are core virtues of governments and other public institutions;
When more people are involved in a process, it will naturally be more representative.

Today, every one of these ideas has been shown to be outright wrong at worst, or, at best, hopelessly naïve. But at the time, they were inescapable. And for the “citizens, policy-makers, technologists, design-thinkers, change agents and media creators” (lol) who were the ChangeCamp constituency, they felt like a genuine, widespread, and necessary evolution in our social and civic life.

You can observe strands of this ideology popping up everywhere around this time:

The Arab Spring (2010)
Occupy Wall Street (2011)
“Crowdfunding” services like IndieGoGo (founded 2008) and Kickstarter (2009)
“Sharing economy” (lol) companies like Airbnb (2008) and Uber (2009)
Massive Open Online Courses (MOOCs, peaking around 2012)
Archive of Our Own (2008), the repository of ~~fan-fiction~~ “transformative works”
“Collaborative mapping” services like Ushahidi (2008) and its precursor OpenStreetMap (2004)
Open Parliament (2010)
Cryptocurrency (Bitcoin, first minted 2009) also has its roots here

In very real ways, we’re today living through the “Age of Participation” envisioned by ChangeCamp and all its fellow travellers. I’d say this age is considerably darker than we envisioned, sitting around those tables at the library one February evening in 2010. But it’s still nice to remember the optimism of that moment, and how excited we were about Change. We probably should have been more specific.

Using the Eleventy Image plugin without a central image folder

2022-05-29T16:00:00Z

🚨 Update, December 2024

Eleventy Image 5.0+ offers a better method for achieving this same result. It’s better for a few reasons:

Relative image paths just work out of the box.
You use standard image Markdown (![alt](img.src)), instead of a non-standard shortcode.
In dev mode, images get processed on-demand, so build times are faster.

I no longer use the custom shortcode method documented below. You can eliminate the special Liquid syntax and get basically identical behavior with just a few lines in your Eleventy config file:

import { eleventyImageTransformPlugin } from '@11ty/eleventy-img';
//...
eleventyConfig.addPlugin(eleventyImageTransformPlugin, {
 extensions: 'html',
 formats: ['avif', 'webp', null], // Pick your own formats
 widths: [640, 928],              // Pick your own srcset widths
 defaultAttributes: {
   loading: 'lazy',
   sizes: '100vw',
   decoding: 'async',
 },
});

Original post follows, preserved for posterity only...

Background

I don’t use a ton of images on my site, but a while ago I wrote about git cherry-picking and included a screenshot. At the time, I just dropped the image into the same folder as the blog post and kept going, but I knew there were a few problems with that:

The original image was huge: more than 2,600 pixels wide, far wider than the maximum width it would ever display at.
All users would download the same 98 kB image, regardless of whether they were on desktop or mobile.

I knew I wanted to eventually add some better image pre-processing with Eleventy Image, the framework’s official plugin. This week I finally got around to implementing it.

Overall it was pretty straightforward, but I ran into a few gotchas and wanted to write them up.

The goal

Here’s what I wanted the plugin to handle for me:

Criterion	Outcome
Automatically resize and optimize images	✅
Automatically support multiple modern image formats	✅
Automatically output `<picture>` elements with `srcset`	✅
No changes to my existing site file structure	🤔

The first three were not a problem; all that stuff is supported by Eleventy Image out of the box. That last one, however, posed an issue.

My site directory structure

Supporting my existing file structure turned out to the be hardest part. Instead of keeping images in a single folder at the project root, I like keeping images adjacent to the relevant markdown files, like this:

src/
  ├─ blog/
  │   ├─ post-title/
  │   │   └─ index.md
  │   │   └─ image-1.png
  │   │   └─ image-2.png

Putting on my information architect hat, I prefer this structure for a couple reasons:

I find it more intuitive to keep related material together.
It doesn’t force me to come up with globally unique names for images; the directory structure itself ensures a unique path.
When writing in markdown, no messing around with "../../../images/file.jpg". You don’t have to remember anything about your directory structure in the moment, you can just write "file.jpg".
This is speculation on my part and not a personal priority, but theoretically, image SEO is probably better if there are contextual keywords in the image path.

Eleventy Image, however, basically assumes that you’ll always specify image file locations from the project root, and that you want to output the processed images to a single centralized directory, something like this:

src/
  ├─ blog/
  │   ├─ post-title.md
  ├─ img/
  │   └─ image-1.png
  │   └─ image-2.png

This is, for the record, a totally reasonable choice that works for many, many people. But I already have something different and I wanted to support it as-is, without tearing up the entire site structure.

The problem

I decided to go with with the Asynchronous shortcode method, and opted to implement the do-it-yourself <picture> variant.

With setup complete, I tested by updating an image file from the traditional markdown image syntax to the new Liquid shortcode equivalent:

- ![Image of a GitHub pull request with a bunch of post-rebase commits that shouldn’t be there](rebase-hell.png)+ {% image 'rebase-hell.png', 'Image of a GitHub pull request with a bunch of post-rebase commits that shouldn’t be there' %}

But when the server reloaded, I got this error:

[11ty] 1. Having trouble rendering liquid template ./src/blog/git-rebase-using-git-cherry-pick/index.md (via TemplateContentRenderError)
[11ty] 2. ENOENT: no such file or directory, stat 'rebase-hell.png', file:./src/blog/git-rebase-using-git-cherry-pick/index.md, line:12, col:1 (via RenderError)

By default, Eleventy Image assumes that the shortcode image path starts relative to your Eleventy inputDir, so it was looking for a file at src/image.png, not src/blog/post-title/image.png. Since there’s no file there, it throws an error and dies.

The solution

I admit this took me quite a long time to troubleshoot, but eventually I figured out how to adjust the shortcode implementation so it would play nicely with my non-standard file directory setup.

This is a simplified version of the code in eleventy.js, with the key changes highlighted:

const Image = require("@11ty/eleventy-img");

async function imageShortcode(src, alt, sizes = "100vw") {

+ // Prepend the image src with the full directory `inputPath`:
+ let imageSrc = `${path.dirname(this.page.inputPath)}/${src}`;
- let metadata = await Image(src, {+ let metadata = await Image(imageSrc, {    widths: [300, 600],+   // Write processed images to the correct `outputPath`
+   outputDir: path.dirname(this.page.outputPath),
+   // Prepend the correct path to the image `src` value
+   urlPath: this.page.url,  });
  return `<picture>
      // picture markup goes here
    </picture>`;}

Now the image plugin can properly fetch image files from their correct location in the file system, and places the processed versions in the equivalent structure in the site build. I don’t have to change my entire image filing practice, and everyone gets properly-sized, optimized images served to their browser, without any extra work on my part.

I’m pretty happy with the results. Most people will get a smaller, faster image now, with no extra work on my part:

Notes and caveats

This is a win overall, but I still see some issues:

The solution here is very specific to my particular needs at this particular time. The changes as I’ve made them to the shortcode implementation are mutually exclusive to keeping a centralized image folder, and it shouldn’t necessarily be an either-or. I imagine there’s a more general solve for this issue; I took it only as far as I needed to and no further.
Without adding some very complicated additional logic, the automated processing is pretty one-size fits all, and you can see it in the visual quality. This is especially true for screenshots, where the original PNG is very high-fidelity, while the downsized and processed one shows some blurriness and artifacts.
Even worse, even though the processed PNG file has way smaller dimensions, it’s nearly 50% bigger in terms of kilobytes served. The .avif and .webp versions are far smaller and a majority of users will be served those versions (WebP is supported by ~94% of browsers, AVIF by ~69%) so this is a tradeoff I’m willing to make. But still kind of irksome.

Implementing dark mode, for real this time

2022-02-21T17:00:00Z

Years ago I wrote a blog post on creating dark mode with CSS variables. At the time it was more about jotting down the basic idea for my own benefit, but I never actually implemented it. A few weeks ago I felt the itch and decided I wanted to finally follow through for real.

I’m pleased to report that the basic idea works well — you’re looking at it right now — but in putting it into practice, I learned a bunch of new things: some quirks of the matchMedia object, setting and getting cookie values, and the finer points of HSL color spaces, among others. So this is to follow up on that original post and elaborate on some of the complexities I had originally hand-waved away.

This post is in four parts:

Creating themes with CSS variables

The strategy here is basically unchanged from my original post: create CSS variables for each colour, plus variations controlled by some selector. In this case, I'm using a custom HTML attribute on the body tag to toggle the theme on and off.

I opted to update my colour declarations to HSL so it was easy to keep things like hue and saturation consistent; I think it makes the colours feel more harmonious overall, both within and between the two themes. I think it’s also slightly easier to reason about compared to hex colours.

Here’s the relevant CSS:

:root {
  --hue: 215;
  --sat: 50%;
  --colorBgPrimary: hsl(var(--hue), var(--sat), 100%);
  --colorFgPrimary: hsl(var(--hue), var(--sat), 20%);
  --colorFgSecondary: hsl(var(--hue), 25%, 45%);
  --colorAccentPrimary: hsl(var(--hue), calc(var(--sat) * 2), 35%);
  --colorAccentSecondary: hsl(var(--hue), calc(var(--sat) * 2), 50%);
}

body[data-theme="dark"] {
  --hue: 200;
  --colorBgPrimary: hsl(var(--hue), var(--sat), 15%);
  --colorFgPrimary: hsl(var(--hue), var(--sat), 90%);
  --colorFgSecondary: hsl(var(--hue), 25%, 70%);
  --colorAccentPrimary: hsl(30, calc(var(--sat) * 2), 75%);
  --colorAccentSecondary: hsl(30, calc(var(--sat) * 2), 65%);
}

Saving the preference as a cookie

I knew I needed some way to save the user’s theme preference in the browser, since the alternatives were bad:

I’d have to pick one theme to be the default, and if the user wanted something different they’d have to toggle the colour-scheme with every page load.
All pages would load with the colour-scheme preference set at the OS level, and either have to toggle on each page load or have no option to switch at all.

So I knew I needed to set some sort of cookie to persist the user’s preference across page-loads and between sessions. Since I only have two theme options, light and dark, it felt like overkill to use a third-party cookie library, so I went looking for a minimum-viable cookie management strategy. In the end, this was my approach:

The first function reads a cookie value, and comes from this Gist by @wpsmith:

function getCookie (name) {
  let value = `; ${document.cookie}`;
  let parts = value.split(`; ${name}=`);
  if (parts.length === 2) return parts.pop().split(';').shift();
}

The second function sets a key-value pair as a cookie. I just hard-coded the options, since my use-case is so limited. This cookie persists for a year (60 seconds, times 60 minutes in an hour, times 24 hours in a day, times 365 days). It also enforces HTTPS and SameSite which is perhaps a little paranoid, given that the cookie value doesn’t contain information of any value. But then again, why not default the most secure behaviour?

function setCookie (key, value) {
  document.cookie = `${key}=${value}; path=/; max-age=${60 * 60 * 24 * 365}; SameSite=Strict; Secure;`;
}

Uglified, these two helper functions add up to just over 200 bytes, so I opted to just put them inline in the <head>.

Building the theme toggler

The actual theme-toggler affordance is a checkbox, which controls a data-theme custom attribute on the <body> tag. By default it’s set to light:

<body data-theme="light">
...
<label id="theme" for="theme_toggler">
  <input type="checkbox" id="theme_toggler" class="visuallyhidden">
</label>

The checkbox has some extra CSS jazz to make it look nice (see below) but it's still just a checkbox. The logic for manipulating that checkbox, however, had a few gotchas:

The window.matchMedia browser API seems undercooked to me. It requires a bunch of hacky string-parsing and generally feels brittle and error-prone.
matchMedia can also change at the OS level at any time, so that’s an event you need to listen for and (optionally) respond to.
Figuring out which default behaviours to fall back to got me a little twisted up. My first implementation was way too complex because I wasn't thinking from the default and building up the complexity from there.

I can’t do much about the quirks of matchMedia, but I can try to explain the theme-switching logic I came up with:

If there’s no theme cookie, get the browser setting, use that theme, and set a new cookie.
If there is a theme cookie, use that theme.
If the user manually toggles the theme, use that theme, and update the cookie.
If the browser/OS-level theme preference changes mid-session, get the new browser setting, use that theme, and update the cookie.
If all else fails, fall back to the default light mode. If the user has turned off JavaScript, for instance, they’ll see the light theme.

Below is the script. I’ve annotated it with comments to explain what’s going on:

// The checkbox
const toggler = document.querySelector('#theme_toggler');
// Caching the matchMedia color-scheme preference as a variable.
const media = window.matchMedia('(prefers-color-scheme: dark)');

// Function to return the system theme. `matchMedia` returns `matches: true` if
// the browser setting matches the media query passed above. It returns a
// string, either "dark" or "light".
const getSystemTheme = () => {
  return media.matches ? "dark" : "light";
}

// Function to perform all the steps involved in setting a new color scheme.
// Accepts a color string
const setTheme = (color) => {
  // set the cookie with the theme preference, as a key-value pair.,
  setCookie('theme', color)
  // update the custom attribute on the body tag with the new theme
  document.querySelector('body').dataset.theme = color;
  // update the state of the checkbox to match the new theme
  toggler.checked = color === 'dark' ? true : false;
  // handle the labelling of the checkbox for accessibility.
  let other = color === 'dark' ? 'light' : 'dark';
  toggler.ariaLabel = `Switch to ${other} mode`;
}

// If there’s a `theme` cookie, use it to set the color scheme
if ( getCookie('theme') ) {
  setTheme(getCookie('theme'))
// ...Otherwise, set the color scheme based on the browser preference
} else {
  setTheme(getSystemTheme())
}
// Listen for changes to the checkbox state 
toggler.addEventListener('change', () => {
  toggler.checked ? setTheme('dark') : setTheme('light');
});
// Listen for changes to the browser preference state
media.onchange = () => {
  setTheme(getSystemTheme())
}

At this point the toggler was working as intended:

On initial page load, the user gets their OS theme preference
The toggle reflects the current theme preference
The toggle can be checked on or off to change that preference
The preference is saved as a cookie and persists across page loads and sessions.

However, it just looked like a checkbox, and I figured I could make it look a little nicer.

Styling the checkbox toggle

I had two priorities with the more elaborate toggle style:

Keep it accessible by keyboard
Tweak the visuals using only CSS

Hiding the default checkbox UI while remaining accessible for screen readers and board navigation seems pretty straightforward, using the “visually hidden” method:

/* Accessibly hide elements */
.visuallyhidden {
  border: 0;
  clip: rect(0 0 0 0);
  height: 1px;
  margin: -1px;
  overflow: hidden;
  padding: 0;
  position: absolute;
  width: 1px;
}

The final step was deciding what the custom toggle should actually look like. I didn’t want to use a hefty SVG or have to figure out a ton of complex animation, so I’ve opted to style using only Unicode emoji — the "High Brightness Symbol" when in dark mode, and the "Waxing Crescent Moon Symbol" when in light mode.

/* Toggler control, absolute-positioned in the top right of the viewport */
#theme {
  display: grid;
  place-content: center;
  position: absolute;
  height: 2rem;
  width: 2rem;
  top: 1rem;
  right: 1rem;
  border: solid 2px transparent;
  border-radius: 3px;
}

/* Toggler focus outline style. The checkbox can still be focused,
   even though it’s visually hidden */
#theme:focus-within {
  border-color: var(--colorAccentPrimary);
}

/* In dark mode, show brightness symbol. In light mode, show moon symbol */
body[data-theme="dark"] #theme::before {
  content: "\1F506";
}
body[data-theme="light"] #theme::before {
  content: "\1F312";
}

Overall, this was a really good exercise in moving from theory to practice, and I’m glad that I (finally!) followed through. Plus I now see some additional threads to pull on, such as moving from just two binary options to a larger selection of themes.

I’ve boiled down the whole setup into a Gist with the most essential moving parts. Feel free to comment there with any questions or refinements.

The books I read in 2021

2022-01-01T17:00:00Z

Not a terribly long list this year but I went back to pick up some previous reads that I'd tried and failed to get through, so I’ll take it. In alphabetical order:

Age of Extremes: The Short Twentieth Century, 1914–1991, by Eric Hobsbawm
A Children’s Bible, by Lydia Millet
Death in Her Hands, by Ottessa Moshfegh
Expressive Design Systems, by Yesenia Perez-Cruz
Filthy Animals, by Brandon Taylor
Foundation and Empire, by Isaac Asimov
Fulfillment: Winning and Losing in One-Click America, by Alec MacGillis
Homage to Catalonia, by George Orwell
Intimacies, by Katie Kitamura
No One Is Talking About This, by Patricia Lockwood
The Pale King, by David Foster Wallace
The Rest is Noise: Listening to the Twentieth Century, by Alex Ross
Seeing Like a State: How Certain Schemes to Improve the Human Condition Have Failed, by James C. Scott
Super Pumped: The Battle for Uber, by Mike Isaac
Vesper Flights, by Helen Macdonald

Notes

A Children’s Bible is so beautiful, and so bleak. Dramatizing climate change strikes me as something that's quite hard to do, but by turning it into a parable about parents and children, Lydia Millet focuses what would otherwise be free-floating anxiety into a terse, brutal narrative. I read it early in the year and while the plot hasn't really stuck with me, the mood remains incredibly vivid.
After saying I didn't really dig Foundation last year, I read the second book in the series anyway, Foundation and Empire. It's modestly better, with, you know, some actual characters. Will I keep reading this series I claim not to like despite myself? Stay tuned!
The Pale King has an understandable reputation for being somewhat impenetrable but once I got into it I was so hooked. The current cultural consensus on David Foster Wallace seems to run from eye-rolls to outright condemnation, all of which I get. But the work: it still sings.
The central idea of Seeing Like a State — that technocratic authorities wind up distorting the communities they're meant to serve through over-determined metrics — explains so much about contemporary social dysfunction at every level. But the book is simply too long and trots out too many repetitive examples to prove the point. Good stuff in the home stretch about metis vs. techne, and the need to respect local knowledge and craft practices over abstract expertise.

How we rebuilt Shopify’s developer docs (again)

2021-11-24T22:30:00Z

March 2025 I recently discovered that the Shopify UX blog was shut down and its archives are no longer online. So I'm updating this post with the full text of the original article. Many of the links are to the Wayback Machine, because Shopify.dev itself has changed again since I originally wrote this.

Last week we unveiled Dark Mode for Shopify.dev, the developer docs hub for the Shopify platform. But that’s just the latest in a series of major design updates we’ve made to Shopify.dev since relaunching the site at Unite.

When we first launched Shopify.dev in February 2020, I wrote about the design process that led us there as a Senior Content Designer on the team. But even back then, we knew that what we’d built was Version 1.0, and that the site would need to evolve as we observed how Shopify’s vibrant community of developers used it in the real world.

Throughout 2020, we saw some things working really well — and some that needed a rethink. So I wanted to revisit this subject to talk about some of the updates we’ve made to the site’s information architecture, and why we made them.

The original Shopify.dev, left, and the redesigned site today.

What needed updating?

In the original design of Shopify.dev, we sorted the site into four main content buckets:

Concepts: Key context about the platform
Docs: References for APIs and other key products
Tutorials: Narrative how-to instructional content
Tools: Software to accelerate the build process

There were lots of things we liked about this model:

It was useful to group together high-level information about the platform in ‘Concepts’ — our quantitative and qualitative research both suggested quality user engagement with that part of the site.
Separating Tutorials (step-by-step tasks that you need to complete) from References (comprehensive descriptions of APIs) was the right move. While these types of docs are often used in tandem, users consult them for different reasons, and a clean distinction allowed us to refine the design experience of each document type.
Breaking out ‘Tools’ into their own standalone section made our tooling more findable and reduced duplication in other parts of the docs. For instance, App Bridge is required for a variety of development tasks on the platform. Having a single home for the library in the Tools section made it easy and consistent to link to.

But as we lived with the site over its first year in the wild, we observed some other things that didn’t work so well:

The ‘Docs’ label was confusing. People thought of the whole site as ‘the docs’ and so a section called ‘Docs’ (the largest section by number of pages!) wasn’t very meaningful to them. We needed to refine some of the language and labels we were using.
Grouping by information type — conceptual information, tutorial information, reference information — made perfect sense on paper but didn’t work as well in practice. Tutorial content, in particular, struggled under this model, because sheer volume made it hard to do second-order navigation. If you ever landed on this page, with hundreds of individual tutorials forming a wall of text, then you felt this pain as you hunted for the particular problem you needed to solve. We experimented with a bunch of design ideas to mitigate this issue but none of them ever broke the logjam.

The Tutorials page of the original Shopify.dev. Dense though it is, this is still only about a third of the page content.

Grouping by information type had a second flaw that was less obvious but became clearer over time as we talked to more developers in the community. What our users needed from us wasn’t just information, but direction. Our docs were good at describing the different parts of the platform and individual developer tasks, but we hadn’t strung them together into a clear narrative. New users arrived at the site thinking, “I just want to build a theme”, or “I just want to build an app”, and we weren’t clearly laying that process out for them. All the pieces of the puzzle were there, scattered around various parts of the site, but we hadn’t been opinionated enough about Shopify’s vision of what a great app or theme looked like.

Based on these observations, we decided our next design iteration needed to better optimize for real-world developer goals, articulate clearer narratives, and lead by example. We also recruited a small group of developers from the community — some newer to the platform, some quite experienced, and from all around the world—to validate some early design concepts. To get as close as possible to the actual experience of navigating a real website, I built a few low-fidelity versions so our pilot users could click around actual IA prototypes. (Shoutout to 11ty and Alpine.js for making it remarkably easy to build these quick-and-dirty experiments.)

Let’s dig into three specific content design decisions we made in pursuit of our goals:

Re-orienting our main navigation from ‘information type’ to ‘developer goal’
Strengthening our ‘Getting Started’ narratives
Scaling up our approach to example code

From ‘information type’ to ‘developer goal’

First, we reorganized the site not by the type of information (concepts, references, tutorials), but by the developer’s goal: What are you trying to build?

Today, there are four basic “things” you can build on Shopify’s platform: Apps, Themes, Custom Storefronts, and Marketplaces. These things can overlap in some cases (and lots of developers work across categories, of course ) but in general, they’re distinct development activities.

For instance, Apps and Themes have different primary audiences. Apps are focused on extending and enhancing the platform for merchants, the millions of entrepreneurs who use Shopify to sell their products. Meanwhile, Themes are all about creating refined online shopping experiences for the tens of millions of customers who purchase things from stores running on Shopify.

Apps and Themes are also built using different technologies. Apps are web applications hosted by developers and written in any language, that interact with Shopify through a set of APIs, while Themes are collections of front-end template code, written in Liquid, that are uploaded to Shopify and run on our infrastructure. Apps and Themes have different developer tooling and design paradigms, and the developers who create them have different business models.

All of this may seem obvious — developers know, intuitively, that Apps and Themes are different things. But it was important to be crystal-clear about how we would recognize and enforce these conceptual boundaries so that this information architecture can scale with the platform. Today there are four ‘Things’ on the platform, but soon there are likely to be more, and we wanted to have some guardrails in place so our docs can grow in a way that’s orderly and predictable for everyone.

The road not taken

This way of organizing the site was not the only option we considered. A strong alternative was to sort our docs, not by the developer’s end goal, but by their stage in a typical software development lifecycle.

This would have led to a site organized similarly to Apple’s dev docs. Apple’s developer hub navigation is built around a typical developer journey:

Discover: Exploring what’s possible on the platform
Design: Defining the desired user experience
Develop: Realizing that experience in code
Distribute: Selling the final software product to end users

There’s a lot to like about this model:

It scales well, in the sense that it can accommodate a wide diversity of software types.
It also scales well in terms of the absolute number of types — this structure works just as well for 3 types or 300.
It proceeds linearly through a typical developer journey, guiding the user through each phase of a successful project.

So why didn’t we take this path?

The first answer is quantitative: we simply don’t (yet!) need to accommodate the sheer scale that Apple does, supporting multiple development ecosystems around multiple hardware and software platforms. That day is coming, but it’s not here yet. (Maybe you’d like to help us get there?)

The second answer is qualitative, and it came from the many conversations we had with developer partners as we were evaluating the existing website and planning this round of design updates. Unless prompted, developers simply didn’t talk about their experience with Shopify in terms of which ‘stage of their journey’ they were in; they talked about the stuff they were building. They had no common language for ‘being in the discovery phase’ or ‘starting the distribution process’; they simply didn’t use that vocabulary. But they could always tell us — usually, with great enthusiasm — what they were building, and how those things worked together to solve merchant problems.

Ultimately, we decided this was the right evolution for the site’s information architecture: Build the Thing. Developers come to Shopify’s platform with an incredibly diverse range of needs, ambitions, experience levels, team structures, and business models. Real-world software development rarely follows a straight, tidy path. What united every developer we talked to, however, was that they had a clear vision of their end goal: to build a great app, or a gorgeous theme, or a unique custom storefront. We reasoned that organizing the site this way would help them get there faster, and keep their eyes on that horizon, regardless of the inevitable bumps in the road or detours along the way.

The exception

API references, such as this page for the productCreate mutation, need to support frequent use and repeat visits.

We did preserve one important division based on information type, however: API references still reside in a dedicated section of the site. By studying the usage data, as well as talking with the development community, we know that references are among the most frequently used parts of the docs. That makes sense: once you’ve worked through a tutorial a few times, you’ve probably grasped most of the basic concepts and know the steps involved. It’s not material you need to revisit much.

But a reference is something you’ll come back to repeatedly throughout your build process, regardless of your experience level. We often heard from developers that they keep the Admin API docs or the Liquid reference open in a browser tab all the time.

As a result, we wanted to ensure that references remain easily and quickly accessible from anywhere on the site. That core principle influenced a bunch of downstream design choices, such as putting the most popular references in a dropdown menu for quicker navigation, and using a shorter, memorable, and easier-to-type URL, shopify.dev/api.

Strengthening our narratives

So our first step was reorganizing the docs architecture around the end goal: build an app, a theme, or a custom storefront. It’s important to know your destination. But what about how you get there?

One of the things developers appreciate about Shopify’s platform is how flexible it is — especially when it comes to building apps. After all, most apps in our ecosystem today are web apps that get embedded in the Shopify admin. As an app developer, it’s up to you to pick your back-end programming language, user interface frameworks, and hosting infrastructure. If you really wanted to write your app in COBOL and run it on a cluster of Raspberry Pis in your basement… well, you probably could, technically! But is that actually the right choice?

In the past, we often weren’t clear enough about what Shopify considers best practice on our platform. It was like we dumped a giant box of Lego on the floor and said, “Go for it.” But we actually do have opinions about how to build for Shopify, and we’re committed to articulating that vision more clearly.

That commitment manifests in a few ways in this latest design iteration.

First, we created more ‘Getting started’ tutorials that are focused on helping you build consistent, baseline implementations. Whether you’re getting started with a minimum-viable boilerplate app, or adding power and complexity to manage product subscriptions, we’re embedding Shopify’s opinionated development paths throughout our docs in a clearer and more consistent way. The goal is that all developers should be able to create a baseline experience representing the state of the art in terms of security, privacy, and performance.

Second, we created sections devoted to literally articulating what we consider to be best practice on the platform. ‘Best practices’ are all the things that we don’t enforce programmatically through the architecture of the product, nor through human-mediated measures like the app and theme review process.

We built out new content about what Shopify considers best practices for many common development activities. The aim is to help all developers consistently build more performant, accessible, and maintainable projects.

Consider guidance like Reduce JavaScript usage. The proper amount of JavaScript for your app to load is “just enough to do what it needs to do”. We can’t tell you what that magic number of bytes is. What we can tell you is what we believe good performance looks like, so that the whole developer community has a common understanding, and merchants and their customers get more consistent, performant user experiences.

Scaling up our approach to example code

With the initial launch of Shopify.dev, we had sorted all our tutorial content into one big Tutorials section. One thing that became clear was that we had a lot of things that were called ‘tutorials’ that were really just a list of examples. Take a tutorial called Use statuses to identify, filter, and manage products. It’s a very long page but its substance is honestly pretty basic: four example API calls to query or edit a product status.

There were two reasons we wanted to cut down on this type of tutorial:

Developers told us that the API reference was a more natural place to look for examples
Maintainability for our internal documentation teams

We’ve heard it loud and clear from developers: good examples are the fastest way to get productive with an API, and those examples should be integrated directly into the API reference. When you’re writing code and referring to a reference doc to complete some specific task, it’s less disruptive to have example code immediately available in that browser tab, rather than having to navigate to a different part of the docs to look up an example. So maintaining that flow state and reducing context-switching for developers was a high priority.

API examples such as this one are now more comprehensive, showing requests in multiple coding languages and sample responses. Some of this content existed before, but it’s now managed in a more scalable fashion.

The second reason was scalability and maintainability for Shopify’s docs team. Those standalone example ‘tutorials’ were fully manually authored, usually by a technical writer working from the API reference. That means that updates were also manual, and required repeatedly reviewing every tutorial with each quarterly API version release to validate that everything still worked. With a few dozen of these tutorials, that’s a hassle, but it’s doable. But now that we’re greatly increasing the volume of examples we provide, it simply wasn’t going to scale.

Shopify’s REST API reference docs are generated by unit tests in the codebase itself, while GraphQL references are generated by the API schema. That means every engineer at the company who works on API functionality documents that feature themselves, in code.

With the redesign of the Admin API references we unveiled in September, all API examples, for both REST and GraphQL, are now created and managed the same way, and reside directly in the reference codebase.

There are a lot of benefits to this new system:

We can distribute the workload of creating example code across every platform engineering team. A small incremental effort by each individual creates a huge increase in value for the developer community as a whole.
Examples are now maintained by the same engineering teams who create the API functionality itself. That means they have a detailed understanding of the business problem the functionality solves, and generally create more realistic examples as a result. Plus, they typically know all the edge-cases, nuances, and potential gotchas that developers need to know about.
API functionality and examples are more tightly coupled in the codebase, so updating the API and updating the code examples can be done simultaneously — often as part of the same pull request—instead of having to do the work in two separate code repositories. Code examples are far less likely to get out of sync with the actual product this way.
We can programmatically test example code for correctness as part of our continuous integration and deployment (CI/CD) pipeline. This was never practical in our old system, where example code was manually authored directly inside Markdown files.

Growing our collection of example code in the API reference is definitely a work in progress, but we’ve already been able to dramatically increase the total volume of example code in our new API references, and there’s much more planned in the coming months.

Documentation is never done

It’s been another big year of change for Shopify’s developer docs, and a big leap for the evolution of our content and information architecture.

Shopify’s platform is growing and changing all the time, and Shopify.dev will continue evolving with it. We’ll keep listening and learning, and we hope you’ll continue to share your feedback with us. But for now, go build the thing!

Previously: How we rebuilt Shopify’s developer docs

Adventures in web archiving

2021-10-09T17:00:00Z

From 2012 to 2018 I worked at Canadian Business magazine. The publishing business itself was crumbling around us, but I still loved the work and the magazine we were putting out. The dwindling budgets fostered a foxhole spirit among the remaining staff; we had to get creative to keep it all going.

The magazine all but folded in 2017, when Rogers Media ended its print edition. CanadianBusiness.com remained online to host a few remaining sponsored editorial packages and a trickle of CP wire stories. The brand was sold to St. Joseph’s Publishing in 2019.

A few months ago we learned that St. Joe’s was going to relaunch the magazine, in print and online. Good news! Then, a few days ago, I heard through the grapevine that the new CanadianBusiness.com probably wasn’t going to preserve the existing site content. Bad news!

I’d known this day would come sooner or later, and I get that the company needs a fresh start to build something new. But I also didn’t want something that I’d worked on for years to be obliterated outright. So, what to do?

Introduction to web scraping

The site is already fairly well-indexed in the Internet Archive’s Wayback Machine. But it’s hard to get a good sense from the outside how comprehensive the archive is, and how systematic the crawl. So I decided to look at how to create my own.

My goal was (and is) to ensure that the Wayback Machine has a complete record of CanadianBusiness.com as it existed in the last days before its 2021 relaunch.

A few years ago I ran across Archive Team, a loose network of “rogue archivists” who do huge scrapes of endangered web content, mostly to preserve through the Internet Archive (although they have an arm’s-length relationship to the IA itself.) Archive Team maintains a list of software for general-purpose web scraping. But the Wayback Machine only ingests Web Archive (.warc) files, which meant straying a little farther into the tall grass.

I ended up installing grab-site, which will crawl a website and produce a series of WARC files containing the results. I started scraping the site on the evening of Monday, October 4, 2021, and it ran for about 72 hours. It traversed 516,858 URLs, and downloaded 26 gigabytes total. A little extra searching around led me to some helpful directions for uploading large web archives to the Internet Archive.

Those WARC files are now freely available on archive.org.

CanadianBusiness.com officially relaunched on the morning of Thursday, October 7. I think it’s a very good looking magazine. Its sitemap lists 27 posts.

Now what?

I’m not totally sure what the next step is. Apparently only approved accounts can put web archives into the Wayback Machine. I’ll need to contact someone at the Internet Archive or on Archive Team to see what it takes.

An important caveat: I have no idea whether I did this right. I followed the documentation and I can see that I created a big pile of data, but given the short time frame and the fairly steep learning curve, it’s hard to know whether what I produced is actually fit for purpose. It is absolutely possible that I’ve produced something that cannot ever be inserted into the Wayback Machine; I just don’t know yet.

I opted to scrape first and ask questions later, and I think on balance that was still the right choice. But it means I’m still figuring out exactly what the necessary steps are. To be continued!

Stray thoughts and observations

The software ecosystem for web archiving feels very wild and woolly. The tooling can be somewhat arcane and intimidating, largely command-line Python programs. Someone of my experience level can puzzle through, but there’s no hand-holding here.
It’s difficult, as a newcomer, to inspect your work and get a sense of whether you’re doing it right. This is partly due to the industrial-grade nature of the tooling; It’s also due to the scale you have to work at for even a modest-sized site — potentially hundreds of thousands of URLs and gigabytes of data. It’s hard to do a dry run and make sense of the resulting output with any confidence.
The Internet Archive itself strikes me as curiously silent on citizen contributions to the Wayback Machine. You can save a single page at a time through the web interface but there’s otherwise very little official guidance on this type of workflow.
All of the above makes this whole enterprise pretty intimidating to get into, which is a shame. I think further democratizing this ecosystem and lowering the barriers to entry might be healthy overall. But obviously it’s hugely reliant on volunteers who are likely already stretched thin. That’s why I chose to donate to the Internet Archive this week. None of my minor gripes about the ergonomics of this process change the fact that the IA does absolutely essential work.

My year of Eleventy plugins

2021-01-25T17:00:00Z

One year ago, I published my first Eleventy plugin (and my first npm package of any kind): eleventy-plugin-youtube-embed. I’d recently rebuilt my personal site using the Eleventy static site generator, and wanted a simple way to add embedded YouTube videos in Markdown. I suspected others might want to do the same.

Quite a few did! Today I know that while it’s possible to make this task easy, it’s far from simple. And that first plugin has since grown into a (tiny) plugin ecosystem of its own, called Embed Everything, which now covers not just YouTube but Twitch, Vimeo, TikTok, and more.

I thought I’d mark this milestone by sharing some of the things I’ve learned over the last year.

The numbers

First, some raw statistics. These figures cover the whole family of plugins.

25,000+: Total number of package downloads since launch.
428: Public repos with one or more of my packages as a dependency.
395: Total commits across all repos.
113: Package versions released.
2: Outside code contributors. (Thanks Leslie and Scott! 🎉)
1: Feature request from Mr. Eleventy himself.

Overall, it’s been really rewarding to see people use something I’ve made. It’s also been hard! In fact, the rewarding parts and the hard parts have mostly been the same parts.

Testing: it’s hard

I had never once written a test before starting this project.

I always knew that I should know more about testing, but (as with so many things) it never clicked until I truly needed it to solve a problem. Which was: I needed to prove, programmatically, that these plugins do what they claim to do, and do so in predictable and reliable ways.

My initial tiptoe into writing tests was simple fire-fighting: I got a couple of GitHub issues filed by early adopters who found the YouTube plugin wasn’t working for them. It quickly became clear that my early haphazard efforts at spotting edge cases was not going to cut it, and I needed a better system. That finally spurred me to make a concerted effort to dig into testing.

I’ve found that learning to write tests (and, to be clear, I consider that learning process very much ongoing) has had the following benefits:

It improves overall project structure
Re-writing these codebases for testing has made them more modular and functional, and therefore easier to read and reason about. Breaking functions down into their smallest constituent parts, so that they’re accessible to tests in the first place, makes them more legible and composable — both for me and (I hope) for other contributors.
It gives you a framework for thinking more deeply
I’m self-taught; getting code to work at all is often a triumph for me. Going back to figure out all the ways things could go wrong always just felt like a drag. But I also never had a framework for thinking about errors. Now, I can sit down and map out expected inputs and outputs, consider possible exceptions, and think logically about how to handle them. It’s not that I couldn’t imagine stuff breaking before; it’s that I didn’t know how to articulate my intuitions. It’s like I could hum the tune but didn’t know musical notation. Now, the known unknowns are much easier to spot and write down.
It helps you move faster
Simple trial and error has two drawbacks: First, it’s unreliable because it’s often un-repeatable, with techniques and approaches varying from person to person and machine to machine. Secondly, and as a result, it just takes a long time. Having tests that can run automatically, regularly, and consistently makes it possible to make changes and updates with greater confidence, and that is so much faster and more productive. There are definitely updates and features in the plugins today that I would not have had the confidence to send out in the wild if not for the enforced discipline of a continuous integration setup.

For lots of people, all these points will be old news, and it’s clear I have more to learn. But adopting even the basics has been enlightening. And if you’re in the same boat I was with testing, I highly recommend diving in.

Versioning: it’s hard

I’m making a good-faith effort to stick to the principles of semantic versioning. But it does feel like semver can be overzealous in a way that’s not necessarily helpful to the majority of end users.

Take any of the Embed Everything plugins: their “public APIs” have been quite stable, mostly because what they do is pretty basic: URL in, embed markup out.

If you’re consuming these plugins as packages from npm, in fact, there are many version releases that would have no visible change whatsoever. In general, we .npmignore test files, the code of conduct, CI config files, and so on, because that stuff really only matters to people contributing to the codebase, not consumers of the package. But those files are also where most of the actual commit action is. So I frequently end up releasing new minor or patch versions that represent no actual change for the vast majority of users.

To me, that doesn’t feel like it’s quite living up to the intended benefit of semantic versioning, which is conveying meaning. Meaning to whom? Open source software has multiple user bases, and a patch release that’s meaningful to a contributor might be meaningless to an end user drowning in Dependabot notifications.

I don’t know how to fix that, and obviously it’s a complex problem. But it’s been on my mind (since I’m kind of drowning in Dependabot notifications).

Naming things: it’s hard

In retrospect, I really wish I’d picked the right naming scheme for the first two plugins. Both the YouTube and Vimeo plugins follow the format eleventy-plugin-{service}-embed. With the later plugins, I changed it to eleventy-plugin-embed-{service}, so that they’d cluster together alphabetically in web searches, in the dependency list in package.json, and so on.

At the time I published the YouTube plugin, I wasn’t thinking about extending the model to other services, so the package name structure didn’t seem that important. The mismatched name pattern has basically zero practical effect, but it just...bugs me. At this point, I think changing it isn’t worth the trouble.

Self-doubt: it’s hard

I’m glad these plugins exist! I’m happy to have contributed them to the Eleventy community, and I intend to keep going. But I definitely also harbour doubts. They include:

Implementation
I’m occasionally haunted by the idea that the fundamental nature of these plugins is...wrong. They work by running a regular expression on Eleventy’s HTML output. Parsing HTML with regex is somewhat famously (though not universally) regarded as a bad idea. I guess the only response I have here is that, given my skills and knowledge today, I’ve made the best thing I’m currently capable of. And yet, I wonder if there’s some dramatically better way — and whether I’d recognize it if I saw it.
Performance and scale
Especially with embed-everything, which aggregates all the individual plugins, I worry that regexing every single HTML file multiple times is not sustainable, especially as Eleventy matures, people’s sites get bigger, and the number of files grows.
Sustainability
Everyone knows open source has a burnout problem. That’s not a problem for me at the moment, but I am aware that keeping these packages up to date requires periodic maintenance and I want to strike a workable cadence for the long term. I’m not really sure what that looks like yet.

Roadmap to v2.0

I do have thoughts about the next major release(s). The top-line feature I’m planning is inline options, so you can change the settings of individual embeds with simple keywords on the same line as the URL in your Markdown.

Here’s what I mean:

<!-- Embed this video with a 4:3 aspect ratio: -->
https://www.youtube.com/watch?v=mT0RNrTDHkI 4:3

<!-- Autoplay and loop this video: -->
https://www.youtube.com/watch?v=mT0RNrTDHkI autoplay loop

The idea is to support features that you likely need to control on a per-video basis, like autoplay or aspect ratio. Some videos require slight tweaks and currently there’s no way to break the mold on a case-by-case basis. I think inline options are the way to handle that.

This represents a breaking change to the existing regular expression structure, which is currently very strict about skipping paragraphs that contain anything other than a URL. But I think the additional flexibility it unlocks is worth it.

My intent is to cut a new major version of all the plugins at the same time, even if the inline options available at 2.0 are limited or non-existent. But I think breaking the RegEx pattern should be done only once across the board, even if the new features come later.

Other things on the radar:

Drop support for Node.js 8. Definitely a 2.0 breaking change.
Support the new aspect-ratio CSS property launched in Chrome v88.
Standardize all the plugins on an embedMethod option (name could change!) to make it easier to better support multiple types of HTML output. For example: iframe, script, lite, oembed, and so on.

The latter two ideas here could be non-breaking 1.x releases too, I haven’t totally settled on how to do it. And all of this is up for discussion — if you have ideas or want to help out, I’m all ears!

The books I read in 2020

2021-01-01T17:00:00Z

You’d think the waking nightmares of 2020 would prompt more escapist reading, but instead I zagged toward nonfiction. I do notice this list of authors is very male: something to work on.

In alphabetical order:

1491: New Revelations of the Americas Before Columbus, by Charles C. Mann
Against the Grain: A Deep History of the Earliest States, by James C. Scott
American Radical: The Life and Times of I.F. Stone, by D. D. Guttenplan
The Dream Machine, by M. Mitchell Waldrop
Fight No More, by Lydia Millet
The Fire Next Time, by James Baldwin
Fortune’s Children: The Fall of the House of Vanderbilt, by Arthur T. Vanderbilt II
Foundation, by Isaac Asimov
Infinite Detail, by Tim Maughan
Lincoln in the Bardo, by George Saunders
Nature’s Mutiny: How the Little Ice Age of the Long Seventeenth Century Transformed the West and Shaped the Present, by Philipp Blom
The Nickel Boys, by Colson Whitehead
The Night of the Gun, by David Carr
No Rules Rules: Netflix and the Culture of Reinvention, by Erin Meyer and Reed Hastings
Something That May Shock and Discredit you, by Daniel Mallory Ortberg
Strangers: Homosexual Love in the 19th Century, by Graham Robb
Team of Rivals: The Political Genius of Abraham Lincoln, by Doris Kearns Goodwin
Uncanny Valley, by Anna Wiener
Utopia Avenue, by David Mitchell

Notes

I don’t have something to say about every book here, but some stray thoughts:

Fortune’s Children is a wonderfully indiscreet history of the Vanderbilts, written by a latter-day Vanderbilt nephew. Tons of detail but always breezy and readable.
The Nickel Boys is a perfectly cut gem. Whitehead’s earlier novels dabble across genres, and here he deploys some of those techniques to give his flinty realism a mythic quality. The result is beautifully haunting.
I read tons of sci-fi as a kid, and yet I never cracked any Asimov, strangely. I can see why Foundation was influential in its time but all its ideas feel out of date now, and the writing isn’t good enough to bridge the gap.
I audiobooked Lincoln in the Bardo, which I think was a mistake. The recording has like 150 celebrity voice actors switching in and out like a radio play; I suspect it works better on the page.
The Dream Machine was out of print for years and very hard to find before Stripe Press reissued it in a beautiful new hardcover volume. Definitely lives up to its billing as a key history of the early computer age, but not a casual read. I stalled out once and came back to it months later. I’m glad I did!

Embed Everything

2020-04-28T23:20:00Z

I recently released an open source Eleventy plugin called Embed Everything. It’s actually seven different plugins and counting, bundled together. I think it has potential!

What it does

The plugin turns URLs into embeds. For instance, drop a YouTube URL into a markdown file, and it comes out as the full thing. So this:

https://www.youtube.com/watch?v=SQIYZ-iKuG4

Becomes this:

Why do this?

I recently re-built my website using Eleventy, which I really like. I think it's a great choice for small personal sites that don’t need to be updated super-often.

One of the reasons I wanted to re-do the site was to ease (back?) into old-school blogging: the low-pressure, digital pastoral sort of thing where you can quickly drop some stray thoughts or share something you found online.

In other words, I want the convenience of “quickly sharing a funny YouTube video on Facebook,” but without the Facebook part. Admittedly, no one will see my funny video because a) they’re all on Facebook, not browsing their impeccably groomed lists of RSS feeds; and b) I’m doing all this out of hermetic self-regard anyway, so who cares.

The point is, Embed Everything makes it about as easy to do that in this bespoke static blogging framework as it is in Facebook, or Wordpress, or whatever. I get to go LARPing as a netizen of the antediluvian blogosphere, but I don’t have to give up the modern conveniences of platform capitalism. And now, neither do you!

How we rebuilt Shopify’s developer docs

2020-02-12T22:30:00Z

March 2025 I recently discovered that the Shopify UX blog was shut down and its archives are no longer online. So I'm updating this post with the full text of the original article. Many of the links are to the Wayback Machine, because Shopify.dev itself has changed a few times since I originally wrote this.

Photo by Nicole De Khors from Burst

Today, Shopify relaunched its developer documentation. This was a long, complex project that required pulling on every UX and development discipline working inside the company, and it’s great to see it finally out in the world. I had the unique privilege of working on this project from beginning to end, so I wanted to share some of the lessons we learned on the way.

The final verdict is, of course, up to the community of thousands of developers who turn to our docs every day. We won’t know for some time whether we’ve truly hit the mark, and as we continue to gather feedback, we’d love to hear from you.

I can, however, provide some background on the origins of this change: why we decided it needed to be done, and how we decided on the changes we’ve made.

As with just about everything we launch at Shopify, our developer docs will continue to evolve as we learn more about how people use them. But the core of this project was about trying to understand what makes a great developer experience.

So how do we know? Well, we asked.

Why change the docs?

First, let’s rewind. What’s the problem we were trying to solve here?

More than 1,000,000 businesses now run on Shopify, and each one has unique needs. Shopify solves many of their hardest commerce problems out of the box — but it can’t be all things to all people. One of Shopify’s secret weapons has always been the large community of third-party developers who build apps, themes, and other integrations to solve problems for merchants. Their hard work makes Shopify flexible and adaptable to just about any special case you can think of.

Shopify has grown a lot over the last few years, and we were finding that our developer documentation wasn’t scaling at the same pace. The growing pains showed up in subtle but increasingly troublesome ways:

Tutorial and how-to content had become mixed in with reference documentation. My analogy would be like using a recipe that doesn’t separate the list of ingredients from the directions. Sometimes this is the right choice! But in our case, it was leading to growing complexity that was making the docs harder to maintain. We haven’t totally solved this problem yet, but we have a clear path forward with this new site.
The old site structure was very deeply nested, which made it hard for developers to browse and discover new and useful features spontaneously. Take something like Kit, the conversational “virtual employee” that merchants can install on their stores to automate common tasks. This is a powerful feature of Shopify and developers can extend it in all sorts of ways. But if you didn’t already know about Kit, you were fairly unlikely to stumble over it, since it only showed up three levels deep in the “Embedded Apps” menu. (Why was it put there? No one could quite remember.)
The app and theme developer docs were struggling in the Shopify Help Center.

The Help Center’s primary job is to assist business owners who are running Shopify stores. Developer content, focused on interacting with the underlying Shopify APIs, was sometimes an awkward fit. Shopify merchants and developers sometimes have related needs — and plenty of merchants do their own coding! — but there are genuine differences between the types of documentation each audience needs. We wanted to set up our developer documentation platform for future growth and evolution by separating these concerns.

Some of these pain points were visible when browsing the docs. Others were internal organizational issues — the little behind-the-scenes dramas and stresses of any fast-growing company. Both types were making it harder and harder to maintain the docs, let alone grow them for the future.
Our developer docs were, in fact, rated fairly well in our annual surveys of the community. There was no crisis that demanded an instant fix. But we could see trouble on the horizon. Documentation that is hard to maintain is documentation that soon gets outdated, duplicated, and disorganized. So we knew we wanted to address these problems sooner rather than later. We knew we needed to do…something. But what, exactly?

The content audit

I started by simply trying to get an idea of how much stuff we were dealing with. Our goal was to get the lay of the land, understand the potential scope of the work, and start narrowing the focus to a manageable, shippable project.

I pulled out my dog-eared copy of Content Strategy for the Web, by Kristina Halvorson and Melissa Rach. It’s one of the definitive books about the craft, and it contains great advice on how to do a thorough content audit.

There’s not enough space here to get into all the gory details of my audit methodology, but it was a combination of manually combing through our codebase, some brute-force web and API scraping, and trawling through our historical analytics. It took a few weeks, but in the end I had an enormous spreadsheet that could answer many of our quantitative content questions quickly.

In all, my initial audit turned up 2,868 pages of material that would be relevant to Shopify’s developer community. It included things like:

Hundreds of pages of instructional documentation in our Help Center
Hundreds more pages of API reference docs. (For the most part, this type of page, describing a particular API resource in detail, is automatically generated straight from the actual API, not manually authored by our technical writing teams)
More than 800 posts on our Partner Blog, many of them covering development topics
Shopify’s Partner Academy, the online learning platform that offers courses and certification, and which contains hundreds of pages of educational material
Polaris, Shopify’s open-source design system
Around 500 open-source GitHub repos.

By correlating individual URLs with analytics data and git commit histories, I was able to fill out the spreadsheet with more detail, and we began to see some pretty clear patterns: which parts of our docs were being revised most actively; which sections were most popular with readers; which ones were performing well in search. I also categorized pages by type (auto-generated API reference pages; human-authored instructional content) and by broad topic (themes, apps, custom storefronts).

At this point, we felt like we had a good map of the territory, but a quantitative content audit can only really give you a snapshot of how things stand today. To figure out what changes we needed to make, we’d need to talk to users.

The research phase

To puzzle through our questions in a more structured way, I turned to our User Experience Research team. One of the great privileges of working at Shopify is having access to world-class UX researchers who love untangling exactly these kinds of thorny, ambiguous questions.

For this project we did two types of research: one-on-one interviews, and a moderated card-sort exercise. In both cases, the goal was to figure out a common mental model for the Shopify ecosystem. If we know how people think about our platform, we can design an information architecture that matches people’s understanding, reinforce the things they already know, and, hopefully, fill any knowledge gaps.

User interviews

Andrew Rajaram, our UX researcher on this project, designed the program and conducted most of the interviews. We recruited two panels: existing Shopify partners who were already experienced with the platform; and developers who had never used Shopify before. We also balanced the list for gender representation, a mix of international markets, and business size, from solo freelancers to people who worked at large firms or agencies.

Andrew drew up a script to run through with each interviewee, so that we’d ask all the key questions consistently. We scheduled the interviews for an hour each by video chat. In each one, there was the interviewee, the interviewer, and a note-taker to observe.

The interviews were instructive. There were a few clear patterns that came through:

1. Most developers wanted to learn by doing

While there’s definitely a segment of developers who thoroughly read over documentation before starting their development process, a majority of the ones we interviewed like to start something quickly and see how far they can get, consulting references and tutorials as they go.

There’s fascinating pre-existing research on this topic, identifying developers as taking either “systematic” or “opportunistic” approaches to navigating software documentation. The systematic ones like to read first, then code. The opportunists start writing code as quickly as they can. It’s worth noting that neither style is associated with “better” code or a more successful outcome; but it does mean that documentation needs to be accessible and useful to both types of learners.

2. Interest in legal documentation scaled with firm size

At mid-sized agencies and up, the interviewees typically asked more about the API terms of service, privacy policies, service-level agreements, and other legal or contractual documentation. This is especially important for those types of companies, because they’re often working on behalf of clients who have their own requirements around licensing and user privacy.

This was one of the valuable new insights we learned from our interview process — one that hadn’t been suggested by earlier research. Once we heard it, of course, it made perfect sense. As a result, the new developer docs foreground things like Shopify’s API Terms of Service, and concepts around user privacy and legislation.

3. Documentation needs change over time

This may sound obvious, but this insight is what steered us away from organizing our documentation around user personas. Often, UX practitioners bucket users by common personas: for instance, if you were building an app for running a car dealership, you’d probably design different activity flows for the people selling the cars out on the lot (a “salesperson persona”), compared to the mechanics working in the service bay (a “mechanic persona”). This technique allows you to make informed generalizations that address common workflows and user needs.

This clear-cut division of labor is less prevalent in software development. At a larger shop, there may be a vice-president who needs to know about the business opportunity, a chief product officer who needs to know about the platform’s high-level technical capabilities, and a team of devs who need to know the API references in detail. But often, those information needs overlap and blur; sometimes, all those “personas” are, in fact, a single individual — just one who’s progressing through the different stages in their development process.

This was what convinced us that organizing docs around personas was a dead end. People aren’t “beginners” or “experts” — they’re people with a problem to solve. We can’t read their minds or pigeonhole them based on (likely erroneous) assumptions about what they want to do at any given time.

This finding led us toward a flatter information hierarchy, increasing the number of topics at higher levels of navigation. Our hypothesis is that this makes it easier for casual users to browse and discover, while motivated users can more quickly drill down to answer their specific questions.

Card-sorting

Our next step was a moderated card-sort exercise. This involved giving developers a stack of about 50 “cards”, each with the name of a documentation concept or topic on it. We asked them to sort these cards into piles that made sense to them, and then label the piles. We wanted to see if we could find common mental models in how people arranged information about Shopify’s platform.

What emerged was surprisingly consistent. Very often the arrangement of piles came out looking something like this:

Basic information about the platform
Legal requirements and responsibilities
Thorough reference documentation
How-to guides, instructional material, tutorials
Software tools, libraries, and SDKs

What was really interesting about this result is that it matched some other research we’d been doing in parallel about DITA, which is an XML-based format for managing documentation, originally developed by IBM in the early 2000s and now an open standard. DITA categorizes instructional material into three broad categories:

Concepts (contextual information you need to know)
References (a detailed description of the thing you’re working with)
Tasks (actions you can take, based on your knowledge of 1 and 2)

The overlap between theory and behavior was striking. People were consistently looking for:

Background information and context (Concepts)
A comprehensive “dictionary” of Shopify’s capabilities (References)
Guides to implementing common workflows (Tasks)
Tools to help them do their work faster

DITA’s core specification doesn’t address “tools”, but the categories were clear enough that we decided we could start prototyping and try revamping the information architecture around these ideas. The result, many months later, is what you see today.

The result

As you can see, the architecture of the site pretty closely mirrors what we learned from the user research. The five top-level “zones” are Concepts, Docs, Tutorials, Tools, and Community (the last of which, for now, only links out to other parts of Shopify, like our developer forums and Partner Academy).

With today’s launch, Shopify.dev has become the primary home of all our developer-facing documentation and educational content, and we see lots of room for further growth and innovation in Shopify’s developer experience.

We feel pretty confident that this new information architecture sets us up to continue growing in an organized and scalable way. But whether it’s useful? Intuitive? Quick to navigate? Clear? That judgement is ultimately up to you.

Next: How we rebuilt Shopify’s developer docs (again)

Rebasing against a feature branch with Git cherry-pick

2020-01-23T02:00:00Z

I’ve been working on a project recently where we occasionally deal with some pretty gnarly rebases. I recently learned how to recover from them much more easily.

This is our setup^[1], which is quite common:

The master branch runs in production
The team contributes to a very active and long-lived feature branch.
We branch off the feature branch and merge contributions back into it.

Anytime there’s an update to master, we have to rebase the feature branch against it. (We rebase on Wednesdays in order not to fall too far behind.) That means any personal working branch checked out before the rebase winds up with a ton of extraneous commits and merge conflicts when making a pull request on GitHub:

This is a screenshot of a real pull request. I’ve obscured my colleagues to protect the innocent.

Sometimes I’ve slogged through and reconciled the conflicts, but it took a long time and felt like a waste of effort. Other times I’ve just given up, deleted my local working branch, recreated the work on a fresh branch, and kept going.

A few weeks ago, my colleague Tiffany Tse taught me a technique that sidesteps this mess with git cherry-pick. Note that this really only works if you pushed up your local working branch to remote (in my case, GitHub).

The method

Here’s how it works:

Check out the rebased feature branch and do a pull so it’s up to date and includes all the rebase changes:

$ git checkout feature/branch
Switched to branch 'feature/branch'
$ git pull
Already up to date.

Delete your local working branch that’s been caught in the rebase:

$ git branch -D my-working-branch
Deleted branch my-working-branch (was 478fe9e).

Checkout a new branch with the same name as the old one:

$ git checkout -b my-working-branch
Switched to a new branch 'my-working-branch'

This is where the magic happens. On GitHub^[2] find the commit(s) you need to rescue from rebase hell. In this example, it’s this one. Use its hash to git cherry-pick just that commit for your fresh branch:

$ git cherry-pick 478fe9e68a71fd706149f95e0719cdd9bdc4197d
[my-working-branch 77d97fc] adding a feature
 Date: Sat Jan 11 13:39:23 2020 -0500
 1 file changed, 3 insertions(+)

Finally, force-push the new branch to remote. Force-pushing wipes out all the rebased junk and re-applies the smaller batch of changes you were making:

$ git push origin +my-working-branch

(The plus-sign +branchname shortcut was also new to me. It’s functionally equivalent to -f.)

In just a few days this has definitely saved me time and hassle, and it was something I’d never seen in many months of periodically searching for how to make this process less painful. Maybe it’ll save you some time too.

Try it out on a toy repo

To help understand the problem and this particular solution, I created a GitHub repo that recreates the problem:

master is up to date.
feature/branch was two commits ahead of master, but has been rebased against master.
my-working-branch is one commit ahead of the non-rebased feature branch. A pull request against the feature branch produces merge conflicts.

Feel free to clone the repo to sandbox the problem and test the git cherry-pick approach to rebasing.

Some argue this whole scheme is a bad idea; I take the point but we deal with the world as it is. ↩︎
...or whatever service you use, of course. You could also comb through git reflog if that’s your thing. ↩︎

Content strategy on the command line

2019-11-21T13:10:00Z

Recently at work I had the opportunity to work on Shopify App CLI, a command-line tool to accelerate the development process for developers building on Shopify’s platform.

For a product content strategist, an all-text user interface is both intriguing and intimidating. I got to write about the process for Shopify’s Partner Blog:

Normally, content strategy is one piece of a much larger design puzzle: content strategists typically collaborate with UX researchers, designers, front end developers, and product managers to ship a new feature. Building a complex graphical user interface requires this kind of collaboration across a wide variety of UX practices.
But what happens when the content is the interface?
That’s the question we’ve been grappling with as we work on the Shopify App CLI, the open-source command-line tool we officially launched last week. The process raised some fascinating content strategy questions and while we haven’t answered them all yet, I’d like to share some of what we’ve learned so far.

Read the full post here.

Create a dark mode with CSS Variables

2019-04-19T21:54:00Z

February 2022: I finally got around to implementing this method for real.

Apple introduced “dark mode” for macOS in late 2018 and now designers are scrambling to duplicate the look everywhere. While Apple’s own guidelines say dark mode is strictly an aesthetic preference, others cite benefits like improved battery life and reduced eye strain.

I'm currently dabbling in my own site redesign, and trying to decide on an updated colour scheme. I was curious how complicated it would be to create my own dark mode.

It turns out: not that hard! But the approach I took relies on CSS Variables, so it requires a little planning. In this post I’ll explain how this minimal proof of concept works. You can skip to the final product if you like.

1. Define the defaults

I’m going to use light mode as the default, and let people switch to dark mode if they want. That means setting the default colours on the :root element:

:root {
  --colorBackground: white;
  --colorForeground: darkslategray;
  --colorLink: royalblue;
  --colorCode: crimson;
}

For now, I'm setting colours for just a few elements. (For a more complex project, this list would surely get longer.)

2. Define a dark colour scheme

The benefit of CSS Variables is right there in the name: they’re variable. You can change them any time and everything that references them changes too. So what we’ll do is change the colour values when they’re inside an element with a .dark class:

.dark {
  --colorBackground: black;
  --colorForeground: lightgray;
  --colorLink: cornflowerblue;
  --colorCode: lightpink;
}

3. Use those colour variables in your design

Now you simply use these variables in the rest of your CSS. Instead of, say, color: darkslategray, you’d write color: var(--colorForeground). For example:

body {
  background-color: var(--colorBackground);
  color: var(--colorForeground);
}

That may feel more cumbersome, but it allows you to be flexible in other ways. Want to change your look a year from now? Just edit those colour names.

In this case, the important part is that it also lets you use code to change colours on the fly.

4. Add a dark mode toggle

We already defined our dark mode colour scheme above. To activate it, we just have to add that .dark class to the page. Since it’s an on-or-off value, let’s use a checkbox to toggle it on or off:

<input id="darkmode" type="checkbox" /> <label for="darkmode">Dark Mode</label>

By default, it will start unchecked. With Javascript, we can tell when that changes and toggle the .dark class on the page’s <body> element:

// Find the checkbox by its ID
document.querySelector("#darkmode")
  // listen for when the box’s “checked” status changes
  .addEventListener("change", function(event) {
    // toggle the `.dark` class on the page’s body element
    document.querySelector("body").classList.toggle("dark");
  });

And that’s it. When the user checks that box — either by tapping on a touch screen, clicking with a mouse, or navigating with their keyboard — the colour scheme switches to dark mode.

The full working example

This Codepen demonstrates the principle in full.

See the Pen Simple Dark Mode with CSS Variables and 3 lines of Javascript by Graham F. Scott (@gfscott) on CodePen.

Getting more elaborate

This is a very minimal example, but you could extend it in several ways:

Use cookies to remember the user’s choice of colour scheme.
Create more complex checkbox styles.
Offer more than two looks by changing the <body> class to, say, high-contrast or bubblegum.
Use a media query to detect the user’s preference at the OS/browser level.

Notes and caveats

Support for CSS Variables is pretty good. As always, use progressive enhancement to make sure your site remains accessible to everyone.

Getting Cron jobs to actually run

2019-03-18T18:43:39Z

Cron is the utility that allows you to schedule repetitive tasks on Unix-like systems. Mostly I use it to schedule backups: I have a little script that sends a copy of my database to Amazon S3 every day. It seems like it should be straightforward, but there’s a gotcha that I didn’t understand for a long time, and I think I’m not alone.

You control Cron by editing your Crontab file with this command:

$ crontab -e

Crontab’s syntax is terse: numbers separated by spaces to indicate time, and then a command to run:

35 22 * * * <command>

For instance, that means “Run this command every day at 10:35 PM”. Not difficult to grasp conceptually. But I always had trouble getting cron jobs to run properly because almost no one tells beginners that the <command> you invoke must be an absolute path, not a relative one.

Here’s a common situation to illustrate what I mean:

You’re logged in to your machine as, say, a user named webmaster.
You have your backup script, backup.sh, saved in your home directory.
You type ~/backup.sh into the command line.
The script runs successfully!

So intuitively, it feels like this Crontab line should work:

35 22 * * * ~/backup.sh

...but it won’t, because when Cron runs, it doesn’t assume that user’s identity. It’s not starting its operation from their home directory, so the relative path doesn’t work.

Instead, you have to use the full, absolute path to the script, starting from the root directory:

35 22 * * * /home/webmaster/backup.sh

I was confused for, literally, years about why I couldn’t get cron jobs to run, and this was almost always why. Many of the popular Crontab tutorials and blog posts out there, even otherwise fairly thorough ones, use absolute file paths in their examples — but they very rarely explain why, or that this is effectively a requirement, not just a convention.

Theatre in 2030

2015-07-29T08:17:00Z

February 2021: A few years ago I was involved in a digital theatre project called SpiderWebShow, which is still going today. In early 2015, the project members were each asked to articulate a vision of theatre in 2030. Our answers appeared as part of Sarah Garton Stanley’s article “Spinning a National Imaginary” in Canadian Theatre Review. I recently stumbled over my answers again and thought it was fun, so here it is.

Theatre keeps fracturing and forking, so there is no coherent vision for theatre in 2030. Instead a thousand new genres are flourishing. Including:

Recap theatre

Actors re-enact episodes of popular television shows, incorporating audience suggestions for script improvements, character motivations and alternate outcomes. Over time these theatrical versions are deviating so far from their source material as to bear almost no resemblance.

Exotheatre

Preparations are underway for the first theatre performance to be produced on another planet. It takes the format of a ritualized memorial to the Soviet Mars 2 lander, the first human artifact to reach the surface of the Red Planet. The performance has an audience of three humans and five robots.

Singularity theatre

Artificial intelligences that have ingested the entire known cultural corpus from the Epic of Gilgamesh to The Real Housewives of Taipei have become prolific authors of theatrical works, with the leading programs having written billions of plays, most of which remain unperformed and unread by humans.

Fake theatre

The audience receives a weekly email describing to them a piece of theatre they could have seen but did not attend. It describes the production in broad strokes, provides a list of specific moments or telling details for credibility, and a unique, algorithmically generated opinion about the production you can share with friends.

Xeritheatre

A movement of touring companies which thrive throughout the western North American dust bowl, mounting quasi-religious revivalist passion plays about water conservation and ecological hubris, and distributing desperately needed bottled water and iodine pills.

Podcast: Web analytics for magazines

2015-02-23T09:00:00Z

July 2022: I recently ran across this recording again and wanted to make sure it was saved in my archive. Some of this advice still holds, although, listening today, it definitely shows its age.

Recently I spoke about web analytics with Tina Pittaway, host of the Magazines Canada podcast, AudioMag. We talked about some of the ways magazines can measure their success online, distinguishing signal from noise, and making sure you’re using metrics to answer the right questions.

Audio

Download MP3 file

Transcript

I’ve lightly edited this transcript for better readability.

Tina Pittaway 0:04
Today on audio mag:

Graham Scott 0:06
We're discovering that some of the things that we think audiences should be interested in, they're just not. And that can be really hard to hear for publishers. But the truth is better than not knowing. Because if you if you don't know, you can't fix it.

Tina Pittaway 0:21
Canadian Business’s Graham Scott on what you need to know about web analytics.

Graham Scott 0:25
There’s nowhere to hide.

Tina Pittaway 0:26
That’s next.

Welcome to AudioMag from Magazines Canada, I'm Tina Pittaway.

Understanding the analytics behind your digital offerings is crucial in trying to better understand how your readers are connecting with you. Analytics lets you find clear answers on what people are reading, how far down they're scrolling and who the frequent sharers are.

But how often should you dive into the data? And just because something doesn't perform well online, what can the data reveal that might point you towards doing it better next time? Sometimes it's as simple as creating a different kind of headline in the one offered on the printed page.

Graham Scott understands a lot about analytics. He's the Managing Editor, Digital at Canadian Business, and teaches The Online Publishing Toolkit at Ryerson University’s Chang School for Continuing Education. I spoke with him following a two-part webinar he presented on this very subject. I began by asking him why it's important for publishers to have a good understanding of analytics.

Graham Scott 1:32
Analytics gives you a way to answer questions. I think when you're doing it, right, you know, so much of the analytics we have now we, you know, we can measure all kinds of things. And in a way, we can measure too many things. Like, when I sit there, and I open up our analytics suite for CanadianBusiness.com, we literally have three or four different analytics platforms that measure everything that happens on the site.

So one of them measures in real time. So I can see what are people reading on the site right now; we have one that is kind of demographics, comScore, which measures things so we can sell the site to advertisers. We have Omniture, which is meant to be kind of the big official one — it's what we base decision-making on and all of that stuff.

And, like, honestly, you can measure pageviews, you can measure the the amount of time that people spend on the page, you can measure how many visitors and how many pages each one goes to. And there are more of these numbers being added all the time. Like, if you if you want it, it is there.

But that's not necessarily helpful if it's just a blizzard of numbers. And what I think the the usefulness of analytics is, is when you have a hypothesis as a publisher, and analytics is a way to help you test that hypothesis and make business decisions.

Tina Pittaway 2:54
What is the really sweet spot for what you're trying to engage the reader with? Or does it depend on the platform that they're accessing? What guides that?

Graham Scott 3:05
I mean, the platform definitely matters. And I think app measurement is a whole other kettle of fish. And one, frankly, that I don't have a lot of experience with. So my experience is related to measuring people's engagement on websites. And on the open web, I would say.

And I mean, the platform matters there because people behave differently if they're at a desktop computer, at their desk, or their laptop. And that's different from how they behave on a tablet or a smartphone. So it is important to keep track of all of that information so that you have it.

The thing that's happening right now is that we're actually kind of in a period of change for what publishers are looking to prove to advertisers in terms of who is on their site. And this is largely driven by advertisers and publishers wanting to show not only that people land and look at a page on your website, but that they are repeat visitors, that they are people who will share your content with their friends on Facebook or Twitter, that they are a desirable demographic group that is reading your site and that they're they're sticking around for a long time.

And the way that the web started, or the way that the monetization of the web started, was with the banner ad, and that was a pretty crude measure, because all we had was: “Did someone look at the site, or did they not look at the site?” And what a lot of publishers are finding now is that they're having to prove engagement at a deeper level. And so they're looking for things like “how far down the page did someone scroll?” and “How long did they spend on the site, and how often do they come back?” And “What is their social relationship to the site, in terms of social media?”

All of those things together are proving not just awareness of a site, but trying to prove that the people, you know that your audience is really an audience, and not just a bunch of random people clicking on on random links. And I think that's particularly important for magazine publishers.

Tina Pittaway 5:22
Why are advertisers interested in those heavier users? Why aren't they just interested in reaching that eyeball for three seconds?

Graham Scott 5:33
Well, I mean, for advertisers, especially brand advertisers, they want to be in a high-quality environment. And that's something the basis of which we've sold print magazines for years: that a subscriber who has paid up for your magazine is considered by many advertisers to be more valuable than someone who is just getting it for free. And magazines that have that kind of premium audience are able to charge a premium.

And the same is true on the web: that people want, you know, they want access to the right audience. And, you know, the promise of the web always was we know exactly who was on the site, like you can go much deeper; you can get much more granular as well and have a clearer idea of what is happening on a website than you necessarily can in the pages of a magazine that has gone to your subscriber.

Tina Pittaway 6:31
I know one of the statistics that you put up in one of your presentations was that something like 40% of the articles at the Globe and Mail are read by only 1,000 people.

Graham Scott 6:42
There's nowhere to hide here. And that's, one of the things that is scary about it — is that for a long time, you know, we were able to talk about these things in fairly abstract ways, like who the audience was, how valuable they were, what their interests were. And now we are getting kind of a warts-and-all picture of the readership in a way that, you know, was not quite possible before. And there are publishers who are certainly finding that, you know, the things that they thought were successful for them, are not necessarily successful on the web.

Part of this has to do with the fact that the kinds of things that work well in a print magazine don't always work well on the web, and it takes a different strategy. And if you just take something from a magazine and put it online, it is unlikely to perform really well.

The other part of this is that we're discovering that some of the things that we think audiences should be interested in, they're just not. And that can be really hard to hear for publishers. But the truth is better than not knowing. Because if you if you don't know you can't fix it. Once you kind of steel yourself to look with a fairly clear view at what is actually working on your site and what is not working on your site, you can have a very clear picture of that. You know what people find engaging, what they love clicking on, what they love sharing with their friends, and that can teach you a lot about your your audience.

I think there's a huge amount of opportunity there. It can be really painful as well. The sensation of opening something up and seeing that it just has not connected with the audience, that they are not interested — that can shatter some some illusions that we clung to, and still do cling to, for totally understandable reasons.

Tina Pittaway 8:34
What are some of the challenges of just too much data?

Graham Scott 8:39
That is a huge challenge. So one of the experts that I've quoted several times on this issue is a guy named Stijn Debrouwere, who is a developer who has worked on the Guardian and several other high profile projects. And the saying of his, that I really love and that I quote as often as I can is: “Analytics are for doing, not for staring.”

And this is something that I've absolutely done: open up the the analytics site, check out the numbers, and if the graph has ticked up that day, I feel great. And if the graph has ticked down, I feel bad. And that is, like a totally natural thing to do, but it's not much use.

What I think people need to focus on is to find a couple of metrics that make sense for their business goals, monitor those, unclench a bit, and calm down about the day-to-day noise in that data. You want to stay on top of it, but it's really hard to think about this stuff in a sensible way when you're captive to every twitch and jerk in the numbers. You need to kind of smooth it out. Look at it, you know, not day-to-day, but look at it week-to-week, month-to-month, quarter-to-quarter. And in that way, you can start to get a better picture, and a more actionable picture, where you can look at what the big trends are, what the big opportunities are for you as a publisher, and then make a decision that's going to actually have a material effect on your business.

That could be all kinds of things. It could be to do with the functionality of the site. It could be to do with the editorial content. There are lots of different knobs you can turn and dials you can adjust in terms of what you are doing day-to-day. And if you've got a question that you want answered, those analytics can give you those answers.

To give you a concrete example: say you have a "Subscribe Now" button on the site, and you've got one at the top of the page, and you've got one at the bottom of the page. And at the moment, it's red, and you want to try it in green, and you have a suspicion that green is going to be a more clickable color. And you'll sell more subscriptions as a result. And that is a hypothesis that you can test. And you can see the results in analytics by, you know, making some changes to the site, observing how people react to that, and seeing whether it produces an uptick in subscriptions.

That is not something that you're going to stumble across by just opening up analytics and staring at the line. What you need is a question, and it can help you find an answer.

Tina Pittaway 11:36
There's also you when you brought up that notion of publishers realizing that readers just aren't interested in certain areas. Could it also help inform publishers that they're just telling a story the wrong way? Or on certain platforms? Like they're not using images or visualization well enough? Is it one of those things where you abandon those kinds of stories? Or whatever it is that that isn't working? Or you try to figure out if you're just telling those stories wrong?

Graham Scott 12:11
I mean, I think that is kind of the kind of question that you need to put to your analytics. Say Story X is not performing to your expectations; you have an editorial hunch that there is an interest out there in it, and you put it up and it kind of dies. It doesn't get a lot of readers spending time on it, it doesn't get socially shared, any of that kind of stuff.

There are so many ways that you could experiment with changing that story itself, or changing the type of story that you do. Maybe it’s as simple as you need to change the style of your headline writing. That is a huge thing that lots of publishers are still figuring out: that the kind of headline writing that does gangbusters in print just dies online. So how could you try selling the story? How is that headline promoting that story? And is it doing the work that you need it to do?

It could also be: are you getting it to your readers in the way that they want it? Is it that it's just not catching on on Twitter, but it's going great on Facebook? If you need to put a slightly different spin on it in on one platform versus another when you're doing that key promotion?

A lot of people struggle with their homepage. If I go to CanadianBbusiness.com and see all the stories that are there, do people land on that page, and then click on something and find something that interests them? Or do they land on the page, and then throw up their hands? Are they finding content that that speaks to them when they get there?

Could you experiment with better images on the homepage? Is there a tweak that you can make to the design of the homepage that will help that reader who gets there get that second click, who will go and read the article, and share the article, and love the article, and sign up for your newsletter, and subscribe and give me money, and do all that great stuff that follows on from it.

And those are the kind of questions that that analytics can help you with in terms of saying like, you know, if you make the images bigger, and maybe illustration works for you better on your homepage, that's a question that you can ask and that I think analytics can can help you get some clarity on.

Tina Pittaway 14:40
For those smaller publishers who don't have a huge staff. What are some of the tools that they can tap into? You mentioned plugins, what are they and how can they help?

Graham Scott 14:51
So something that I've done a little bit of writing on recently is plugins that are designed to serve related content on your site.

You see these all over the place, like at the end of articles, often you'll scroll down and there'll be a couple of tiles, and they will say, you know, “Recommended for You” or, you know, “Other Stories You May Like”. In many cases, these are third-party plugins that are quite inexpensive to implement, sometimes free, and, usually, technically, not terrifically difficult to implement on a site.

In many cases, they can help you in a way that is low effort on your part, increase the time that people spend on a site. So say someone gets the end of your article, you know, it's a, it's a great piece of writing, and they really enjoy what it what it has to say, and they might be open to reading more on your site, if you can get a couple of links down at the bottom to stuff that is relevant to them. So they've read, they've just read something really great, and they want more, it will help you kind of automatically find related articles on your site, and it will suggest them to them.

And second thing it will do is it will find the things that are already popular on your site. So the things that attract the most pageviews. And it's likely that those are quite appealing. And it'll suggest those as well.

And so a couple of services, you know, one is Outbrain; if you're a larger publisher, Taboola is one, that, that I believe that a paid product, Contextly is another one. And there's even something as simple as the Facebook recommended box, which takes copying and pasting one little piece of code. And it will start recommending things that your friends have recommended on Facebook to every visitor who shows up.

That kind of thing can have a really measurable effect on the amount of time people spend on your site, how often they come back, and making sure that they find something they like when they get there.

Tina Pittaway 17:12
It's both exciting and terrifying all at once.

Graham Scott 17:15
It certainly is. But you know, the nice thing about it is that if you have a question, you can implement a change fairly quickly, and see results fairly quickly.

And, you know, the best publishers who are doing this are doing it all the time — just a constant series of optimizations that they're doing. And sometimes, you know, it produces, you know, half a percent more pageviews or more unique visitors or things like that. If you do 10 of those things in a row, 10 Little fixes that make a measurable increase, that starts to add up to something real. And not all of them are going to work. But that's okay.

One of the nice things about the web, that was not necessarily true in print, is that, with a monthly or bimonthly magazine that comes out six to 12 times a year, you've got to get it right, and you've got to get it right every single time. Whereas on the web, you know, you've got another chance the next day — there's always another train coming. The stakes, in many ways, are higher, but they can be lower, in terms of the individual actions that you can take. If something you try doesn't stick, stop doing it.

Tina Pittaway 18:38
Absolutely. That's there's so much there to chew on. Thanks so much, Graham. All right. Thank you. Graham Scott, Managing Editor, Digital at Canadian Business.