I’ll be introducing some breaking changes in the next major version of Ohm and I’d like to make the upgrade path as smooth as possible. So I’ve been investigating patterns for specific “migration” and “compat” packages in the JS ecosystem.

I found a few interesting examples. Most of them aim to support incremental upgrades, allowing you to migrate to the new API piece by piece. This isn’t really a concern for Ohm. Mainly I’m interested in making it easy for to folks to try the new version behind a feature flag, and easily revert to the stable version if they run into any bugs.

Anyways, here’s what I found —

react-router-dom-v5-compat

Instead of upgrading and updating all of your code at once (which is incredibly difficult and prone to bugs), the backwards compatibility package enables you to upgrade one component, one hook, and one route at a time by running both v5 and v6 in parallel. Any code you haven’t touched is still running the very same code it was before. Once all components are exclusively using the v6 APIs, your app no longer needs the compatibility package and is running on v6.

This package enables React Router web apps to incrementally migrate to the latest API in v6 by running it in parallel with v5. It is a copy of v6 with an extra couple of components to keep the two in sync.

Once you’ve converted all of your code you can remove the compatibility package and install React Router DOM v6 directly.

@vue/compat

Looks like a similar approach to react-router-dom-v5-compat — it includes new version, alongside compatibility features:

@vue/compat (aka “the migration build”) is a build of Vue 3 that provides configurable Vue 2 compatible behavior.

The migration build runs in Vue 2 mode by default - most public APIs behave exactly like Vue 2, with only a few exceptions. Usage of features that have changed or been deprecated in Vue 3 will emit runtime warnings. A feature’s compatibility can also be enabled/disabled on a per-component basis.

zod‘s subpath versioning

Zod takes a slightly different approach — rather than a separate package, it includes the new version as a submodule:

The general approach:

• Zod 4 will not initially be published as zod@4.0.0 on npm. Instead it will be exported at a subpath ("zod/v4") alongside zod@3.25.0 Despite this, Zod 4 is considered stable and production-ready
• Zod 3 will continue to be exported from the package root ("zod") as well as a new subpath "zod/v3". It will continue to receive bug fixes & stability improvements.

Sometime later:

• The package root ("zod") will switch over from exporting Zod 3 to Zod 4
• At this point zod@4.0.0 will get published to npm
• The "zod/v4" subpath will remain available forever

Another approach is a package intended to be used in conjunction with the new version, with extra functionality that useful during the migration.

ember-compatibility-helpers

Provides flags for features in Ember, allowing you to write code that will work with whatever version the consuming application is on. This addon is intended to help addon authors write backwards/forwards compatibility.

The flags are replaced at build time with boolean literals (true or false) by a Babel transform. When ran through a minifier (with dead code elimination) the entire section will be stripped, meaning that the section of code which is not used will not be added to production builds - zero cost compatibility!

jQuery migrate

Upgrading libraries such as jQuery can be a lot of work, when breaking changes have been introduced. jQuery Migrate makes this easier, by restoring the APIs that were removed, and additionally shows warnings in the browser console (development version of jQuery Migrate only) when removed and/or deprecated APIs are used.

💬 Any others I should know about? Send me an email or respond on Bluesky or Mastodon.

The first few times it happened, I wasn’t mature enough to learn much from it. But eventually I discovered a little trick — something that helps me set my ego aside and actually listen.

The details aren’t all that important, but to set the scene: a Principal Engineer I worked with had called me out for being overly negative in a meeting. He pointed out that, as a senior IC, I should be mindful of how my actions could affect those around me.

In hindsight, he was spot on; but in the moment, it felt mostly wrong and totally unfair.

I spent the rest of the day in a miserable mood. In the evening, I was still stewing, and considering how to respond. I think I even started drafting a rebuttal email (I know, I know).

At some point I googled “how to respond to negative feedback” or something like that. Honestly, I’m not sure if I expected to find something useful, but I did.

The advice that helped me: look for what’s true.

Almost any time you get serious critical feedback, it will feel unfair. There will be things your critic got wrong, or important context that they didn’t mention. Your instinct will be to focus on those aspects, to find as many reasons as possible to dismiss or minimize the feedback.

Instead of focusing on what’s wrong with the feedback, ask yourself what’s right. Sure, your critic is totally off base, but is there a kernel of truth there?

It’s a simple trick, but I find it incredibly useful¹. By reframing things, I’m able to diffuse my immediate, emotional reaction. Once I’ve done that, it becomes much easier to see things from an objective perspective.

I don’t really want to get into the reasons, except to say that I’m under no illusion that my support (or lack of it) makes any difference to those companies. For me, it’s much more about reducing risk¹, and about supporting smaller companies whose interests are more aligned with mine.

Anyways! I wanted write about my progress here, in case it’s useful to anyone else who’s also considering this.

(For a more in-depth discussion of the various alternatives, I found European Alternatives and Getting off US tech: a guide to be quite helpful.)

Web hosting: Hetzner 🇩🇪 + Bunny 🇸🇮

This was a pretty easy one. My web site has been hosted on Dreamhost for 20-odd years, and a few years back I started using Cloudflare as a CDN and for SSL termination.

I now have a VPS with Hetzner, and am using Bunny as my replacement for Cloudflare. I’m extremely happy with both:

• The Hetzner VPS gives me a lot more flexibility than I had on my Dreamhost shared hosting plan, and it costs about the same. It’s also cool that my VPS is just down the road in Nuremberg.
• Bunny is a drop-in replacement for most of what Cloudflare can do. Their web UI is also much more pleasant to use than Cloudflare’s, and their custom support is excellent. I pay them $1/mo (the minimum charge).

Mail & calendar: Fastmail 🇦🇺 + iOS Calendar

I’ve also been using Gmail and Google Calendar for almost 20 years — since shortly after they announced Google Apps for Your Domain.

This was one of the most difficult decisions. I evaluated Soverin and mailbox.org, but they both have pretty rudimentary web UIs. Also, I have a huge volume of mail (20 years worth) and use search *a lot*, and I couldn’t figure out a setup that could really replace Gmail for me.

I considered Proton, but since it’s end-to-end encrypted, I assumed that search performance would also be a problem.

I ended up choosing Fastmail. Although they’re an Australian company, their servers are located in the US. Since privacy is not my main concern here, I considered it to be an acceptable tradeoff.

Migrating my email to Fastmail was really easy (they have a migration assistant) and I’m perfectly happy with their web client and iOS app. There was really nothing that I missed from Gmail.

I did have a few issues migrating my calendar, mostly related to notifications from other Google Calendar users. The problem was two-fold:

1. Fastmail only did a one-time import of all the calendar events.
2. Since I still had a Google Workspace account, any changes from other Google users would directly update the calendar event, rather than being sent by email.

It took me a while to debug this, but I eventually realized I could disable Calendar in my Google Workspace. Everything has been working fine since then.

On my laptop, I use the Fastmail’s web UI for my calendar, but on my phone, I’m using the built-in iOS calendar. I used a configuration profile to do this.

For a single user, Fastmail costs €5/mo.

Video calls: Whereby 🇳🇴

After I moved off Gmail and Google Calendar, I need to find a replacement Google Meet. This was surprisingly difficult; I find a lot of video chat apps (e.g. Discord, Cal.com) to be significantly worse than Meet.

Finally I tried Whereby and have had nothing but positive experiences. I’m currently paying for their Pro account (US$7.50/mo).

Web browser: Vivaldi 🇳🇴

Not much to say here. I got tired of Chrome’s deceptive messaging about ad privacy and about how careful I had to be to stay signed-out.

Vivaldi is a Chromium-based browser, and after tweaking a few settings, it’s basically a like-for-like replacement. I’ve been happily using it for 6 months now.

File sync: Self-hosted

I used to have all my important digital records in Dropbox. Early last year, I replaced it with a self-hosted setup:

• A Samba share on a Mac Mini (running Debian) in my basement.
• Unison for syncing my laptop to the server.
• Tailscale + Owlfiles for access on the go.
• A nightly cronjob that backs up to a Hetzner Storage Box. (See TIL: Backups with Borg and borgmatic for more details.)

Maps: Organic Maps + Apple Maps

I’ve been using Apple Maps for navigation for a few years now; I find the UI to be nicer, and for some reason, Apple seems to be better at knowing about road closures in Germany.

For other things that I used to use Google Maps for (e.g. looking up opening hours of stores), I find Organic Maps to be surprisingly good. I’m not sure I’ll be able to completely remove Google Maps from my phone though.

Office: 🤷

The biggest gap for me so far is that I haven’t found a good replacement for Google Docs and Google Sheets. Luckily, my needs aren’t that big; I’ve been getting by with LibreOffice Calc for spreadsheets, and Obsidian for most of the other stuff.

What’s next?

To address the elephant in the room: yes, I’m still using iOS and macOS. I did pick up a Nothing phone last year, and at some point I’d like to try installing one of the deGoogled Android distros on it. I may also pick up a cheap desktop machine and try running Linux for a while, let’s see.

I also haven’t tackled photos yet. I’m still paying Apple for iCloud photo storage, and also using Google Photos. I’ve heard good things about Immich but haven’t tried it out yet.

💬 Questions? Send me an email or respond on Bluesky or Mastodon.

Yesterday, it was an insight from one of my favourite GC papers, A Unified Theory of Garbage Collection, which helped me solve a tricky problem.

ohm’s incremental parsing

I’m working with a team that’s using Ohm to parse text documents and render a rich text version in ProseMirror. The goal is bidirectional updates: changes in ProseMirror should propagate to the text version, and vice versa.

Ohm supports incremental parsing, which means that if you parse some text and then make a small edit, it can quickly reparse by reusing portions of the previous result.

It also supports limited form of incremental transforms. You can define an attribute, which is kind of like a memoized visitor, and the attribute value for a given node will only need to be recalculated if the edit affected one of its subtrees. So you can easily implement a form of persistent data structure, where each new value (e.g., an AST) shares a bunch of structure with the previous one.

the problem

Using this machinery, I tried making an pmNodes attribute that produced a ProseMirror document for a given input. When the text document is edited, it produces a new tree which shares a bunch of nodes with the previous one.

Then, my plan was to construct a ProseMirror transaction that would turn the old tree into the new one. To do that, it’s helpful to know which nodes appeared in the old document, but not the new one.

My first implementation of this was equivalent to tracing garbage collection — after each edit, I walked the entire document, and recorded all the nodes in a Set. The difference between the sets told me which nodes had died.

But this kind of defeats the purpose of incrementality — if you have a long document and make a small edit, we should be able to process without visiting every node in the document.

the solution

Then I remembered A Unified Theory of Garbage Collection, a 2004 OOPSLA paper by some former colleagues of mine:

Tracing and reference counting are uniformly viewed as being fundamentally different approaches to garbage collection that possess very distinct performance properties. We have implemented high-performance collectors of both types, and in the process observed that the more we optimized them, the more similarly they behaved — that they seem to share some deep structure.

We present a formulation of the two algorithms that shows that they are in fact duals of each other. Intuitively, the difference is that tracing operates on live objects, or “matter”, while reference counting operates on dead objects, or “anti-matter”. For every operation performed by the tracing collector, there is a precisely corresponding anti-operation performed by the reference counting collector.

This was the answer I needed! Rather than visiting all the live objects, I wanted to only visit the dead ones, and reference counting would let me do that.

So I added a way of maintaining a reference count for all the nodes in the doc. When we produce a new document, we decrement the reference count of the old root node (it will always be 0 afterwards). So we recursively decrement the ref count of its children, and so on. This gives me exactly what I wanted — a way to find all the nodes that were not reused, without having to visit most of the nodes in the doc.

Together with my friend Mariano, I’ve been working on the book for just over two and half years. And (big surprise) it was a huge learning experience! While everything is still fresh in my mind, I wanted to capture some of the things we learned here.

The clichés

Let’s start with the obvious stuff — the advice you’ll find everywhere, the things everyone wants to tell you when they hear you’re writing a book.

You won’t get rich

Don’t do it for the money. Or the fame. Or any kind of immediate payoff, really.

I don’t think I could have pulled of a project like this if I didn’t find it intrinsically rewarding. The book was something we wanted to exist in the world. Early on, I decided that even if nobody cared about it, I wouldn’t regret doing it.

It’s a marathon

It will take longer than you expect, and then some. As with running, picking the right pace is important. This is especially true if the book is not a full-time project (as it wasn’t for us).

It ended up being very seasonal for me. There were times (maybe ~6 months in total) where the book was my main focus; the rest of the time, it was second or third on my priority list.

When it was my main focus, I tried to put in four or five hours of deep work on most weekdays. In the other periods, I tried to keep a regular schedule: sometimes a day a week, or — when I really need the momentum to get a new chapter out — an hour a day.

Some of the techniques I wrote about in Getting things done (in small increments) came in handy.

The surprises

This would be a very long post if I tried to capture everything that we learned while writing the book, but a few things stuck out as being a bit more surprising than the others.

Feedback is…complicated

We had both read Write Useful Books and were determined to take its advice to heart. We had early reader conversations and built up a list of interested beta readers. This was definitely helpful — it confirmed that there are people out there who would be interested in the book. And it definitely gave us a better idea of who our potential readers might be.

But, while we’re very thankful for the feedback that we got, it wasn’t quite as clarifying as we’d imagined. Maybe it’s because of the subject matter, or maybe we should have done something differently. But I think Write Useful Books underplays how much you still need to rely on your own taste and intuition to guide you.

Explorable < explanation

Our original idea was to make the book quite interactive, and include explorable explanations. We did include some interactive elements, but they’re not really essential to the book. Honestly, we didn’t put in the time that would have been required to make them really great.

The thing is, we found a much bigger payoff in focusing on the explanation. What’s the point of this chapter? What is the right order to introduce the concepts? What should we move to another chapter, and what should we cut altogether?

The time we spent improving the core content of the book felt so valuable, and compared to that, it spending time tweaking the interactive parts just didn’t seem worth it.

Constraints are helpful

From the beginning, the book was planned to be (a) self-published, and (b) online-first. This meant that we could do almost anything we wanted — but in retrospect it was maybe too much freedom.

We spent a lot of time thinking about some basic things, like:

• How should the chapters depend on each other? Are they intended to be read in order?
• What structural and typographical elements should we use — e.g. sidenotes, callouts, etc.?
• What will readers do with the code? Can they execute it in the browser?
• How does the code in each chapter build on the previous one?

We ultimately came up with a structure we like, and we might use a similar structure if we write another book. But we spent a lot of time thinking about things that had nothing to do with WebAssembly.

When I compared this to previous writing experiences — my master’s thesis and a handful of academic papers — I realized how much value there is in having a fixed structure. Academic papers have a template and a page limit, along with many conventions that reviewers and readers expect. Within those boundaries, there’s a bit flexibility, but the overall structure is pretty fixed.

If you’re thinking of writing a self-published book, my advice would be: pick a book that you like, and use its structure as a template. Focus on the core material that you’re writing about, and worry about the other stuff later (if at all).

I could write another thousand words on things that we learned from writing this book. But would it help anyone? I suspect that — like being a parent — you won’t really understand until you’ve made the mistakes yourself.

Are you thinking about writing a book? My advice: go for it! You won’t regret it.

I was writing a lot of JavaScript code and constantly having to wait for code reviews from a team in Los Angeles. On one of my trips to Mountain View, I thought: screw it, I’m going to LA to meet those bastards.

I didn’t ask my manager, didn’t tell the LA team I was coming — just booked a flight, went into the office, and introduced myself.

And you know what? They were incredibly welcoming. They offered me a desk, we had lunch together, and they took me out for drinks that night.

I never had to bug them about code reviews again. One of them (thanks James!) started reviewing my PRs first thing each morning. Not long after that, they added me to the OWNERS file.

While I was booking that flight, I had a thought: Alan Kay lives in LA.

I’d always wanted to meet Alan. I was a big fan of Smalltalk and his more recent work on the STEPS project at VPRI. So I sent him an email:

Hi Dr. Kay,

I’m a software engineer at Google on the Chrome team. I’ve been thinking a lot about tools for web development, specifically dynamic development tools that support exploration, experimentation, and immediate feedback. Your work is a big inspiration in this area, and I think it’s a shame that most environments still can’t do what Smalltalk could 30 years ago.

I’m normally based in Munich, but will be in Google’s Venice Beach office for the next couple of days. I’d love to chat with you about some of my ideas. I’m sure you’re busy, but if you have 30 minutes to spare I’d really appreciate it. I could meet anywhere in the LA area that would be convenient for you.

And you know what? He responded the next day and said he’d be happy to come meet me for lunch. We had a great conversation, and he later introduced me to Alex Warth, one of his “most brilliant grad students ever” who also worked at Google.

Alex and I stayed in touch, and two years later, he asked me to join him at a new research group Alan had started. It was basically a dream job; I spent three years there working with some amazing people¹, many that I’m still friends with today.

Don’t wait. Sometimes you can just make it happen.

💬 Want to leave feedback? Send me an email or respond on Bluesky or Mastodon.

These days, my views are much more nuanced. I try to adapt my style to the situation. The code I write, and the process I use, depends a lot more on what the goals are. Am I trying to bang together a quick prototype to learn something? Or am I fixing a bug that might affect hundreds of thousands of users? My approach would be completely different in those two scenarios.

Years ago, I read Edward De Bono’s Six Thinking Hats, which describes a framework for problem solving and creative thinking. The idea is that you can “put on a hat” to deliberately adopt a specific mode of thinking. It’s a bit corny but (imo) there’s a useful idea there.

Maybe this applies to different styles of programming, too? What “coding hats” do I use?

Captain’s hat 🧑‍✈️

The captain’s hat is about doing things by the book: being careful, moving slowly and deliberately, following the proper procedures. This is attitude I’d adopt if a mistake could crash 8.5 million systems.

With the captain’s hat on, I’d focus on small, focused, independent commits — things that are easy to roll back if a problem occurs. I’d make sure my PRs include tests. I’d write descriptive commit messages and make sure the code is reviewed.

This is the closest thing to what I used to think “the right way” was. What I missed back and then — and it’s a big miss — is that this isn’t the right approach for all situations, not even close.

Scrappy hat 🐕

A few months ago, I was sharing a small prototype with a friend. Before I showed him the code, I felt the need to give some disclaimers: “This is deliberately written in a very minimal style”, “I was going for the simplest possible thing that could work”, stuff like that. Eventually I started thinking: I wrote this with my scrappy hat on.

The scrappy hat is about keeping things lean — taking the straightest possible path to your goal, and skipping the ceremony. No code reviews, minimal tests. Sometimes you just need something concrete to discuss; with the scrappy hat on, you can quickly throw something together and decide later if it’s worth keeping.

This is pretty close to the default mode that most small startups operate in. It’s the hat of the “minimum viable” and the 80-20.

MacGyver Hat 🛠️

The MacGyver hat is all about getting a result. Sometimes you don’t care what it takes, how messy the code is — you just need to figure out if something is possible, whether it’s worth spending more time on. It’s the quick-and-dirty, any-which-way, by-hook-or-by-crook mode.

I’ve used this hat before when I’m doing performance work. Maybe I have five different ideas to make something faster. They’d each take a day or two to implement cleanly, but if I don’t care what the code looks like, I can hack each one together in an hour. Once I know something has promise, I’ll put in the work to do it right.

Chef’s hat 🧑‍🍳

The chef’s hat is all about presentation. You want the code to look beautiful, in addition to serving its purpose. There are few real-world situations that truly call for the chef’s hat, but sometimes it’s just what you want to do.

Sometimes I’ll use this in a time-boxed way — once I’ve got the basic functionality working, I might “put on my chef’s hat” for thirty minutes and see if I can clean the code up a bit.

Like the captain’s hat, this one is sometimes mistaken for The Right Way, and is often overused. It’s easy to convince yourself that you’re doing something valuable, making the code more readable and maintainable. But often you’re just polishing for its own sake.

Teacher’s hat 🧑‍🏫

With the teacher’s hat on, you’re concerned about what your code communicates, and not so much on what it actually does. You might want to use this if you’re writing example code for an open source project, or writing a blog post to explain a concept.

Maybe you skip error checking, because it obscures the main thing you’re trying to show. Or you use more descriptive variable and function names than you otherwise would. You don’t pick the most efficient solution, you use the most understandable one.

Look, I’m not trying to convince you to adopt my framework. I won’t be doing corporate training workshops anytime soon. But I do think it’s worth being deliberate about what mode you’re operating in, and asking yourself if a different approach might be worth trying.

💬 Want to leave feedback? Send me an email or respond on Bluesky or Mastodon.

Related:

• Thorsten: Surely not all code’s worth it
• Jimmy: Discovery coding

If you want to actually do research like this, how should you go about it?

There are a couple of obvious strategies. One is to look for unsolved problems — say, “addressing the scale-up problem in visual programming”. Another strategy is to solve an old problem with new techniques: “X, with CRDTs”. Both of these lead to something that is obviously novel, but it might not really be useful.

Another path

But there’s another strategy, which Alex and I took with Ohm. I don’t think it’s groundbreaking, or even original, but it’s worth talking about because (imo) it’s a less obvious way to do research.

The idea is this: aim for practical utility first, by building something that addresses an actual need that you have. You don’t need an unsolved problem, but there should be something unique about your solution. In a sufficiently complex domain, there’s a good chance that will lead to publishable research.

The core ideas behind Ohm came from Alex’s experience writing grammars with OMeta (which he also created). The abstract of our first publication gives a good overview of what makes Ohm unique:

Parser generators give programmers a convenient and declarative way to write parsers and other language-processing applications, but their mechanisms for extension and code reuse often leave something to be desired. We introduce Ohm, a parser generator in which both grammars and their interpretations can be extended in safe and modular ways. Unlike many similar tools, Ohm completely separates grammars and semantic actions, avoiding the problems that arise when these two concerns are mixed.

Ideally, whatever you build is something that helps advance your broader research agenda. When Ohm was created, I was working in Alex’s research group at CDG/HARC. A lot of our research involved new programming languages and DSLs — so Ohm was valuable to us even if it hadn’t directly led to publishable work.

What can you publish?

That first Ohm publication, Modular Semantic Actions, is an example of the most likely outcome of this sort of research: a systems paper. There are many opinions on what makes a (good) systems paper, but I particularly like this one by Kayvon Fatahalian:

Good systems papers often put forth an argument of the flavor:

“We are looking to achieve the goals A, B, C, under the constraints X, Y, Z. No system existed to do so when we set out on our efforts (otherwise we probably would have used it!), so from our experiences building systems in this area, we can distill these experiences into a specific set of system requirements. These requirements may not be obvious to readers who have not spent considerable time thinking about or building applications in this space.”

(From What Makes a (Graphics) Systems Paper Beautiful.)

But there’s another interesting thing that can happen. Once you’ve built a working system and put it to authentic use — i.e., used it to solve actual problems that you have — you’ll have a totally different perspective on the problem domain.

You’ll likely uncover other interesting research questions, things that weren’t obvious to you before. Those topics will naturally be less crowded, and you’ll have insight and expertise that other people in the field don’t have. On top of that, you now have a working system that you’re an expert in, which you can use for experimentation and validation.

This is what led to our second proper publication, Incremental Packrat Parsing. It’s unlikely we’d have written it if we hadn’t spent two years building Ohm and working on live programming environments. Alex had also been using Ohm in a class he was teaching at UCLA, and incremental parsing solved a real problem that he had with the interactive course materials.

And since we had a codebase that we were intimately familiar with, it didn’t take long to prototype our solution. As I recall, the core technical contribution in that paper was conceived and implemented on a single, two-week trip I made to LA in 2016.

Indirect impact

Building something useful is also a great way to have impact in the wider research community. Patrick Rein, from the University of Potsdam, adapted Ohm into a Smalltalk library called Ohm/S, leading to at least one publication. In Daniel Jackson’s group at MIT, a number of projects have used Ohm, including Déjà Vu and Geoffrey Litt’s Wildcard. Ohm has also played a part of several of Geoffrey’s projects at Ink & Switch, including Potluck and Embark.

Some objections

It won’t work for me! Perhaps not. I’m not saying it’s easy or that it’s the best approach to take. My only claim is that it can work.

It’s obvious! Maybe it is. It certainly used to be fairly common in traditional research environments, both in industry and academia. It doesn’t seem as common these days.

I suspect many people outside these environments don’t even consider it, because it doesn’t feel like research.

What do you know? Honestly, I’m not an expert. I’ve been following the HCI and PL literature for ~20 years, but I’ve only published a few “real” papers². I’ve had a handful of workshop papers, and have reviewed papers for CHI, Onward!, and LIVE. So, take my advice for what it’s worth.

💬 Feedback? Send me an email or respond on Bluesky, Mastodon, or Twitter.

Thanks to Alex Warth for helpful feedback on this post, and for being a fantastic collaborator and co-author.

Corel

My first real internship was on the Linux team at Corel in the summer of 2000. I was super excited about this, for a couple of reasons:

• I had grown up using CorelDRAW.
• I was super into Linux and open source at the time. (As in, I read Slashdot religiously and stayed home on Friday nights to recompile my kernel.) So an internship where I got paid to work on open source code was a dream!

I spent the first summer working on Wine, which Corel used for the Linux versions of their products (CorelDRAW, WordPerfect, etc). Not only did I learn lots about doing serious development on Linux, but I also learned a ton about the Win32 API.

One of the highlights of the summer was getting to attend the Ottawa Linux Symposium. For those that don’t know, OLS was a pretty amazing conference. Here’s how Slashdot described it:

If you’ve got some kick-ass new code to add to Enlightenment, go talk to Rasterman. He’s right over there. Want to discuss something you need added to the kernel, and want to show off your source to Alan Cox? He’s sitting down hacking 2.4 while discussing Scooby-Doo. One of the great things about the Linux community is the ability to contact developers first-hand, especially though E-mail. Ottawa Linux Symposium is a testament to the fact that it works just as well in real life, too.

After my internship was over, I stayed on as a part time employee during the school year, working on the Corel Linux desktop team. I was a full-time intern again the next summer, but things were…different. Most of the Linux team had moved to another company (Xandros), and those of us that were left ended up working on the FreeBSD port of the .NET CLR.²

Eclipse & SWT

In the fall of 2001, I joined a company called Object Technology International (OTI) for an internship on the Eclipse team. OTI was a subsidiary of IBM that was best known for producing VisualAge Smalltalk. They’d also created Eclipse, and at that time, they were just gearing up for the first public release.

Interestingly, many people on the team didn’t have any experience with open source. I remember explaining the basic development process to someone on my team: “So you run diff, generate a patch, and then send your patch to the mailing list…”

The team I worked on was responsible for SWT, the widget toolkit used by Eclipse. This gave me my first exposure to a few different native toolkits: MFC, Motif, and GTK. The main thing I worked on was the CoolBar widget, which earned me the nickname “Coolio” from the team’s TL.

Compared to Corel, the environment at OTI was much more like the stereotypical tech company. While we didn’t have free lunch, we did have a well-stocked kitchen with great breakfast cereals (I was a Corn Pops man), and a pool table and game consoles in the basement. I also remember how at the start of my internship, they apologized that I wouldn’t get my own office — I’d have to share with another intern.

J9 VM Team

The following summer, I returned to OTI to work on the J9 VM team. This was the same team who’d built the Smalltalk VM for VisualAge Smalltalk, and J9 was (at the time) a Java VM mostly designed for embedded applications.

One of the most memorable parts of this internship was how much Smalltalk code I wrote! Large parts of the VM were written in a kind of portable assembly language, implemented as a Smalltalk DSL. For each supported platform, there was a “backend” that would translate the portable assembly to native assembly for that platform.

One of the projects I worked on that summer was getting J9 running again on HP-UX (for PowerPC I think, not PA-RISC). The first step was getting the code generation working. As I recall, the HP-UX/PPC backend was only partially implemented.

After I got the code generation working, it was a weeks-long process of fixing crash after crash. The first goal was just to get j9 -version working, and then to run a simple hello world program, and eventually to get a bigger benchmark like SPECjvm running.

Another project I worked on was for PalmOS. As I recall, J9 supported a kind of static linking on PalmOS, where the app bytecode was bundled together with the virtual machine into a native PalmOS app. When done naively, that resulted in some pretty big executables. So my task was to figure out how to make them smaller.

J9 again: memory management

In winter 2003 I did another internship on the J9 team, this time working on memory management.

As I mentioned, J9 was originally designed for embedded applications, but there was a push to make it work better for server applications. A big thing that needed to change was the garbage collector. So my colleague had taken on the task of writing a brand new memory management framework for J9 and a high-performance, parallel garbage collector. My job was to help him out any way that I could.

At the time, I had zero experience with garbage collection, but I picked things up pretty quickly. One of the first things I learned: since the GC needs to traverse every live object and a good portion of the internal VM data structures, any kind of memory corruption (like a dangling pointer) is likely to manifest as a crash in the GC. As we joked: “Always blame the garbage collector.”

So one of my first big tasks was to work on a heap verifier, aka “GC check”. The idea was to have a command line option (-Xcheck:gc) that would verify the state of the heap, and other GC-relevant VM structures, before and after every garbage collection. That way, when a crash got sent our way, we could pretty easily tell whether it was something caused by the GC or not. It was basically a perfect starter project.

It was an 8-month placement, but it worked out so well that I ended up spending the next four years there — working part time while I finished my degree, and then accepting a permanent job in 2004. I stayed there until 2006, when I got the urge to go back to school and do my master’s in human-computer interaction.

I learned an incredible amount of things from my time on the J9 team. Not only about memory management and systems programming, but about good engineering practices. I remember being struck by the size and scale of the build and test lab: 25m² or so, packed with all kinds of different hardware — from beefy rack-mount servers to Pocket PCs and Palm Pilots. You’d arrive in the morning to find out you’d broken the build on some architecture you’d never heard of until then.

Looking back, I really can’t believe my luck in getting to work on something so technically challenging, but also high impact. Getting to help build a brand-new parallel GC framework from the ground up — as an intern! With one other person!² Pretty amazing.

One of the things I really miss from this period of my career was how frequently I got thrown into the deep end. I joined teams that I was not at all qualified for, and was handed projects that required expertise that I didn’t (yet) have. And this was totally expected!

💬 Feedback? You can send me an email or respond on Twitter.

When you’ve been doing the same thing, with the same guys, for more than 13 years, it’s easy to get a little attached to that structure. So, a few weeks ago, I was a little annoyed when only four of us showed up.

If you’re not familiar with 5-a-side: the game is pretty flexible. Ten players is obviously ideal, but up to 14 is ok — one or two players on each side can take a short break to catch their breath. 4-on-4 works, and even 3-on-3 is alright in a pinch. But 2-on-2? Nah.

At least, that was my first reaction. The four of us bummed around for five or ten minutes while we discussed what we should do. Maybe just head home, and go to bed early?

In the end, we decided to set up a small playing field with some water bottles marking the goals. Then somebody suggested that we should make it a bit more challenging: instead of shooting the ball between the bottles, what if the goal was to knock the bottles over? Another idea: “…and maybe you get two points if you knock them both over.”

The four of us played this game for over an hour, until our legs gave out. And, you know what? It was the most fun I’ve had playing soccer in a long time. I was convinced that anything other than proper 5-a-side, the way I was used to, just wouldn’t be worth it. And I was totally wrong.

Sometime last year, I noticed that I was hardly doing any work on side projects anymore. And when I thought about why that I was, I realized it was because all my side projects felt like work.

I wrote about this in my last post, Taking Learning Seriously:

One of the best decisions I’ve made in the past few years is to stop treating side projects like work projects.

Previously, I tended to work on things that were either useful, novel, or valuable (i.e., could possibly be turned into some kind of business). It’s almost as if I was trying to satisfy a hypothetical Hacker News commenter.

I also felt compelled to make defensible engineering choices on my projects. Like using the appropriate language for the problem, and reusing existing code whenever I could, rather than reinventing the wheel.

It was another instance of unconsciously adopting a restrictive set of assumptions, telling myself that if I wasn’t done “right”, it wasn’t worth doing at all.

And guess what — when I decided to let go of those assumptions, I started having fun on my side projects again.

💬 Feedback? You can send me an email or respond on Twitter.