Own Your Operations

Sat, 18 Jan 2025 00:00:00 +0000

Own It

You need to own your operations. Now.

In Just Buy it Already I made the claim that it nearly always makes sense for companies to buy, rather than build, solutions that support their products. For personal projects, this can also be true. I think that still holds, but I also stated that things change. Sometimes the catalyst for reconsidering those purchases is a need to reduce expenses. Sometimes it’s because a company can optimize a self-hosted solution better than a managed offering.

And sometimes, especially for personal projects, it’s simply about control.

I have a strong sense that in the coming years, those that start to own their operations more, now, will be stronger¹ than if they do not. I cannot point to any specific reasons for this² and I’ll admit this isn’t a well-formed thought, so I’ll try to flesh out the idea with some examples of what this means for me.

Blogs

For 10 years I self-hosted this blog on a small VPS. And it was good. When I started paying for Github Pro, I learned that I could take advantage of Github Pages. Since this blog is powered by Jekyll it was trivial to migrate and shut down my VPS; I saved a few extra bucks each month. This service provides me with the benefits of Github’s infrastructure (global distribution, HTTP caching, etc.) but that likely doesn’t make a dent in my site’s performance given that a good month brings me 100 visits.

Tangential to this, because Github Pages doesn’t provide analytics, I’ve stuck with Google Analytics (GA) for years. It was the only Javascript I used on my static site, but two things bothered me about it: the free tier makes you wait at least 24 hours to see any new traffic and ad blockers likely diminished its usefulness.

I removed GA last month because I’m going to take back self-hosting this blog. And I’m starting by building a simple system to count web hits.

Email

I am loathe to run email for anyone ever again. It’s a never-ending arms race between you and viruses/spam. But for myself? Nowadays, I am seriously considering it, because I want control of my email. I don’t want to give Google/Gmail any more information about me and my activities; I don’t want to be the product. Paid offerings from outfits like Proton are better, in that regard, but still come with significant privacy concerns, especially for those that may be at risk from actions by state actors.

So I will begin the process of running my own mail server. This requires a lot of preparation and planning but it is important to me that I have complete control over the content and transport of my messages. Also, all those domains I bought for all those projects I’m totally gonna finish need some DNS love to ensure they’re not being used for spam.

Caveat

I’ve been working in web operations for a long time now. I recognize my experience affords me a better starting point to take ownership of my personal operations than others; I have the knowledge and have internalized the level of effort required to do so. This stuff ain’t always easy. As I work through my own needs I will write about them and share my solutions, in case others find them useful.

Footnotes

1. I don’t think this is the word I’m reaching for, but it will suffice, for now.

2. It’s more that there are multiple reasons but I need to spend more time thinking through them.

Generative AI, Trust, and Expertise

Thu, 05 Dec 2024 00:00:00 +0000

Lots of articles have been written about generative AI, and this is one of them.

I started scribbling this short post as a bit of prediction, but I think instead I’ll offer it as a small hope: the increased use of generative AI will lead folks to reevaluate what trust, scholarship, and expertise look like.

Before Google started providing AI-driven search results, I didn’t fully trust the top 3 entries it returned. Though they were generally good, I’d still check the second and third pages for additional content, just in case. I don’t immediately accept output from a single search engine query and I’m certainly not going to trust guidance from disembodied software acting as an agent providing potentially life-threatening suggestions about the edibility of wildlife.

Broadening the set of information I consider helps me locate both corroborating and contrarian content I can use to build knowledge. To ask better questions. To refine scope. To improve my research. This is scholarship. An important point here is that use of AI does not, itself, provide understanding. Coupled with a bit of rigor and critical thinking it can support the development of understanding.

For a long time, I think people have implicitly trusted technology companies’ products and services, to the point of deference. My evidence here is anecdotal: friends and colleagues that have told me they don’t look at Google search results “below the fold”, for example. But the more folks experience the limits of generative AI, the more they will seek additional sources of information. Inevitably they will identify human experts.

It’s here that a bit of hope (optimism?) sneaks in: we will increasingly seek out more local sources we can trust (generative AI being the global foil) and build strong communities whose foundations are supported by expertise.

Legacy Systems Support Greenfield Work

Fri, 10 May 2024 00:00:00 +0000

Legacy systems make new development possible. Much like the decomposition of this year’s generation of plants will support next year’s growth, our experiences with existing systems fuel greenfield projects we’ll take on in the future. What you know now is foundational to what you create later.

Crucially, operating these systems encourages us to consider how we deliver care for them because like humans, our systems may not always be healthy. An ailing legacy system teaches us how to identify degraded performance and confirm restoration. This background is necessary for designing tomorrow’s robust (though likely still susceptible) systems.

Embrace your legacy systems and learn from them.

This post was inspired by a number of ideas, not the least of which is this quote from Donella Meadows:

[M]astery has less to do with pushing leverage points than it does with strategically, profoundly, madly, letting go and dancing with the system.
Donella Meadows

Senior Engineers Don't Grow on Trees

Thu, 09 May 2024 00:00:00 +0000

Senior engineers don’t grow on trees, so they say. That is, you can’t simply walk outside and pluck one from a low-hanging branch and reap the benefits of all that experience.

Except, they do. That is, if you consider that trees take years, decades even, to mature and you think of a tree as an organization that fosters an engineer’s growth over time.

Senior engineers come from somewhere, but not overnight.

Start investing time and energy into yours. Be patient. And reap the benefits of all that experience.

AI is not Understanding

Sat, 02 Mar 2024 00:00:00 +0000

AI, specifically large language models (LLMs), are not a replacement for understanding or experience.

Imagine you ask your favorite LLM for a set of recipes that adhere to a certain theme, say, Thai cuisine. You may receive a collection of instructions, including the necessary ingredients and steps to prepare them, but this is not enough to prepare the dishes adequately. So many important answers that affect the execution of your meal are outstanding:

Where can you source the ingredients?
Should ingredients be fresh? Or can any be dried/frozen/processed?
How does one manage the preparation and cooking of the recipes to minimize the amount of time and resources (energy, tools) required?
Is it useful to prepare anything in advance?
Should multiple people participate in preparing the meal? What does coordination look like?

However, if you are practiced at cooking, if you understand basic (and perhaps some advanced) techniques, you are more likely to be successful in making your Thai plates. You may even be able to identify improvements in a recipe or patterns across them, to make better use of your time. Your practice supports your understanding of the art of cooking. That understanding undergirds your experience.

Without that practice, understanding, and experience, any answer an LLM provides is superficial. But in your hands, it can be incredibly useful. Because you can make sense of it.

All this is to say that LLMs are simply a tool. But a powerful tool that can augment work we do, when leveraged by those with the necessary experience to reason about the output it generates. Now, by no means, am I implying that LLMs should not be used by those without experience. In fact, I think LLMs can augment anyone’s work, at different levels. Part of building experience is learning to ask different/better questions and observing the results. One of the benefits of LLMs is that they are trained on large corpuses of text [1] so there is a high probability that some of the answers to your prompts will contain new information that you can use to explore topics more deeply or in novel ways. But remember to always seek out multiple primary sources to refine and support your understanding.

[1] Caveat: I am not addressing the quality of training sets nor the potential for them to contain biased context or incorrect/misleading information. There are more qualified folks in the world that can speak on that topic.

Rhythm is Gonna Get You

Fri, 01 Mar 2024 00:00:00 +0000

In bed

Throw the covers on your head

You pretend like you are dead

But I know it

The rhythm is gonna get you!

Gloria Estefan, Rhythm is Gonna Get You

It’s Friday. The week is winding down. We order pizza and wings and fries. We tune in to some trash TV and tune out from the world for a few hours. We do this every week. It’s our rhythm.

Except this week.

For the past 7 days, a few of us have been sick. Our rhythm is off. It’s funny how being out of step throws the entire cycle into relief. None of it seems important; not the day, the pizza, or the TV. It’s fun, of course, but I could care less if we miss any of it today.

Note-taking

Fri, 23 Feb 2024 00:00:00 +0000

I take notes daily. Copious notes. All the notes.

Why?

The obvious answer is “So I don’t forget.” I began taking notes in earnest after my first annual job review, where neither I, nor my manager, could recall all the important work I had done. I felt like I had missed an opportunity to show off my accomplishments and I never wanted that to happen again.

Really, though, I take notes so that I don’t have to remember.

At any point in time, I likely have several thoughts or ideas bouncing around in my head. I can compartmentalize really well, which feels like I am managing those ideas effectively, but in reality, I’m only deferring those thoughts for some random time. Those times when I’m nowhere near an instrument to capture them (like the shower, washing dishes, etc.). If I can capture them when they’re (relatively) fresh, I can purge them from my mind, coming back to them later.

In short, taking notes helps me focus.

It’s an implicit focus, most times, because I have fewer things on my mind. But that tends to lead to supporting explicit focus later.

How?

When I realized that I take notes to conserve mental energy, the “how” took on a new shape for me. And my tool choice changed as I refined my needs.

I started with Notepad. No organization, no linking. Just raw notes shoved into a virtual “crazy” drawer on my C: drive.

I used Evernote for years, primarily because I could sync notes across all my devices. If I were in the middle of scrubbing pots and an idea flashed, I could hastily dry my hands and tap it out on my phone. Search helped me locate past notes, but the quality of the results was variable, depending on the quality of my notes. Sometimes, I didn’t quite know what I wanted to write down, so a placeholder would be useful. Sadly, Evernote didn’t have a mechanism like wiki linking that allowed for future notes, but I trudged on.

For a few years, I used Roam Research. Part of my note-taking includes daily TODOs to help me organize myself and Roam defaults to a TODO-driven model. That seemed like a good fit. I especially liked that I could create wiki-style links to future notes and hop to them later. For a long time, they had no mobile option, and I missed that, but I grew accustomed to it. In a way, not having a mobile app felt liberating, given how terminally online we all seem to be. When they did release a mobile app, I used it, but found it less compelling than my Evernote days. Despite tagging support, something about Roam Research felt lacking, in terms of how notes could be organized. Yes, it behaves like a graph, but that felt a bit too flat for my needs.

Enter Obsidian. I was aware of Obsidian when I chose to use Roam Research, but I wasn’t interested in self-hosting my notes (I don’t think they had a sync offering at the time). As my dissatisfaction with Roam grew, I took another look. I’ve always been a fan of truly text-based documentation (like Markdown and Graphviz) because it is durable; it’s legible in its raw form and endlessly style-able. I really like that in addition to creating any folder structure I want, links between notes neither require nor depend on that structure. And I can easily create future notes via wiki link-style syntax. I’ve been thinking a lot about privacy lately so I’ve opted not to purchase Obsidian Sync, choosing instead to host my notes in Github (which I already pay for to host code). I have to manually sync from time to time, but since I’m using git every day, doing so feels like a natural extension of my current workflow. I also have been using the Templater plugin to build some light, custom workflows. In total, I am very satisfied.

The Next Tool

I can’t explain it, and I know I’m not alone in this, but I always feel complelled to check out new note-taking tools. I don’t know what they might look like, but I’m certain when they arrive, I’ll dabble.

Mary Shelley's Invention

Tue, 06 Feb 2024 00:00:00 +0000

Invention, it must be humbly admitted, does not consist in creating out of void, but out of chaos; the materials must, in the first place, be afforded: it can give form to dark, shapeless substances, but cannot bring into being the substance itself.

Mary Shelley, Frankenstein (1831 edition)

Change Management in Practice

Wed, 31 Jan 2024 00:00:00 +0000

Recognizing hazard and successfully manipulating system operations to remain inside the tolerable performance boundaries requires intimate contact with failure.

Dr. Richard Cook, How Complex Systems Fail

Change management procedures are borne from a fear of change. The controls related to this “management” afford an avoidance of understanding proposed changes, bolstering that fear.

As a group makes time to understand changes and their expected impact, they learn to describe them more effectively. This leads to less fear of change. And as each change is implemented, the group has an opportunity to calibrate their descriptions (models).

Then, as the group’s collective confidence grows around change events, when failures occur, reactions from fear are attenuated (they’re never dissipated fully). Curiosity gets a foot hold and learning gets a leg up.

Care for Software

Tue, 30 Jan 2024 00:00:00 +0000

I used to agree that every line of code is a liability. After reflecting on this, I see software less as a liability and more a responsibility. Meaning, care is required.

At all stages of software’s lifecycle.

We care as we develop, care as we operate and scale, and care as we deprecate.

Good care requires lots of resources.