Nothing of substance, but enough to once again actively take part in this wild and beautiful medium that is the web; enough to let 47 more blog posts (a blog post being anything I can be bothered to find a proper title for), 22 weeknotes (which taught me that keeping a regular pulishing cadence is not for me), 72 notes and then a few odd pages more follow.

I'm enjoying myself tremendously if I get around and find a moment to put my thoughts down (at times, admittedly, that is quite a big if). And in case somebody else happens to read this: thanks for being along for the ride. :)

So, back in the days, when commuting to work was some kind of bad default most of us were stuck to for no apparent reason, we merrily programmed forms, using a happy little in-house component library. And like any proper component library, this particular one came with a special company-custom button, for merely slapping CSS classes on a plain HTML <button> element and calling it a day is beneath the dignity of serious enterprise APIs.

Back then the company employed a small tribe of shamans (they called themselves designers, but I suspect mostly just because they had rent to pay and shaman is not considered a cromulent job title in an enterprise email signature) who in their unerring wisdom handed out guidelines for the enlightening and overall betterment of the world. These guidance foresaw that buttons shall come in many variations. Amongst them such that were to be used for finalizing actions, and such for destructive actions (with a red background, as if to menacingly say click me, oh user, if you dare), or those that looked like an ordinary link, but clicking them only performed sideffects on the very same page (the only page to be exact, because, like all civilized folks in 2010s, we built single-page apps). And, last but not least, the garden-variety of for all other commands of secondary importance that the toolbar of an enterprise application might want to offer to be invoked.

Meanwhile, a different tribe, the sages of software architecture, in their sisyphean quest to herd the developers (which you can herd pretty much the same way as you herd a glaring of wild cats), became a bit agitated. They feared the notion of creative interpretation of the shamans' guidelines. To prevent anarchy by means of committee steered standardization, one of their young'uns was charged with the creation of the aforementioned library, so that all code bases, just like all happy families, in the end would be alike (the Anna Karenina principle of API design), and each button that would save a form might have resembled something like this (and made damn sure that nobody did something crazy like just using a button!):

<special-company-button id=saveCmd variation=finalize>
Save
</special-company-button>

Everything hummed along quite fine, and the garden-variety, to the suprise of no-one, was by some margin the most common variation. That was why in the initial implementation it was chosen as the default value for the variation. That was what naive younger me thought at least, until that fateful day, when the maintainer of that special-company-button component, with the impeccable reasoning that the most important thing on any form to do was to finally submit it, decided that the default variant henceforth should be finalize. No big deal. A one liner, changed, committed and released as a minor version somewhere between a late standup and the lunch break.

Well, semantic versioning, my sweet behind... No amount of reasoning helped to roll that change back. So, all usages of that component without specific value set (suffice to say, quite a few...) had to be touched again.

Joshua Bloch, the mind behind some of the most used APIs in the Java ecosystem, has a great talk on How to Design a Good API and Why it Matters (PDF) and a condensed set of maxims for API design, one of which I felt particularly strong that day (and ever since):

Public APIs, like diamonds, are forever. You have one chance to get it right so give it your best.

Good defaults reduce cognitive load, and make the most common use cases easy. The best details are those where you (most of the time) don't have to spend brain cycles on. And even seemingly small semantically incompatible changes in widely used public API have a huge impact. Defaults ought to be trustworthy. Primitive value changes are something that a type system usually does not catch (unlike renaming a package or class, or changing a method signature), updating feels smooth and leave you with a silently broken UI at run time. To drive the point home: Default values are public APIs. Handle with care.

The purpose of the event is to advocate for improved maintainability through separation of concerns. Generally speaking, this is possible only if document structure is handled by HTML, and styling and design by CSS. Removing or commenting CSS code and references, as done during CSS Naked Day, leads to displaying only the document structure as governed by user agent styling of the respective HTML code, demonstrating said separation and maintainability, and appearing “naked.” (A website or app that mixes concerns by using presentational markup would still look at least partially styled.)

In order to practice what I generally preach, I will participate as well. I've implemented the following little script for it (which should have me covered for the following years as well).

function isNakedCssDay() {
    let year  = new Date().getFullYear();
    let begin = new Date(`${year}-04-09 00:00 UTC+14:00`);
    let end   = new Date(`${year}-04-10 00:00 UTC-12:00`);
    let now   = new Date();
    return now >= begin && now <= end;
}

function removeStyling() {
    document.querySelector("link[rel=stylesheet]").remove();
    document.querySelectorAll("style").forEach(style => style.remove());
    document.querySelectorAll("*").forEach(el => el.style.cssText = "");
}

setTimeout(() => {
    if (isNakedCssDay()) {
        removeStyling();
    } 
});

For a preview outside of the event, or on other websites, you can use the removeStyling() function also as a bookmarklet.

type LookupKey = string | number | symbol;

type LookupTable<
    LookupKey extends (string | number | symbol),
    ReturnType
> = {
    [key in LookupKey]: ReturnType
};

Let's suppose you need to handle commands that you receive via some message channel.

const COMMANDS = [
    "a",
    "b",
    "c",
] as const;

type CommandName = typeof COMMANDS[number];

With the as const assertion the type of the array COMMANDS is narrowed to a readonly tuple of the assigned values instead of an array of string that would have been infered without the assertion. And by using an indexed access type, you can then create a discriminated union of the legal key names, that is derived from the values of the array. With this type then you can create a specific lookup table:

type CommandHandlerTable = LookupTable<
    CommandName, 
    () => void
>;

const COMMAND_HANDLER: CommandHandlerTable = {
    "a": () => {},
    "b": () => {},
    "c": () => {},
};

What is nice is that this list of legal keys serves two purposes at once: at compile-time to check for exhaustiveness, as well as at runtime to make validations on the actual objects. If you assert that clients might send malformed messages, you need to check all objects that are supposedly commands, before actually executing code that assumes that what ever is send has property which is a legal key in any given lookup table. You can do this with a type guard like the following:

type Command = {
    type: CommandName
}

function isCommand(message: any): message is Command {
    if (message && message.type) {
        return COMMANDS.includes(message.type);
    }
    return false;
}

Now a the implementation of a handler might look like this:

function execute(message: any) {  
    if (isCommand(message)) {
        COMMAND_HANDLER[message.type]();
    }
}

So by simply adding a new value to COMMANDS the compiler will enforce that a handler is added to the COMMAND_HANDLER lookup table, as well as derive the discriminated union of legal keys to the table without having to touch the CommandName type. The execute function doesn't need to be changed, just because a command is added. And if you are willing to forgo the compile-time safety guarantees, this also makes it relatively straight forward to enable the dynamic extension of a system, although in the latter case you'd need some additional logic to prevent "undefined is not a function" exceptions.

This is a small system. Except under extreme circumstances (huge data base, no supporting software), such a system could be produced by a good programmer within a week or two.

The remark about the effort, which is actually only an insignificant aside in the paper, caught my eye. Two weeks is the typical length of a sprint, so most working programmers will have an intuition about the amount of work that is on average fitted into such a timeframe. Here is what would supposedly have been a reasonable expectation in 1972:

The KWIC index system accepts an ordered set of lines, each line is an ordered set of words, and each word is an ordered set of characters. Any line may be "circularly shifted" by repeatedly removing the first word and appending it at the end of the line. The KWIC index system outputs a listing of all circular shifts of all lines in alphabetical order.

The following implementation, straight from that spec, took less than 20 minutes, which included 10 minutes of reading up on the various configuration options of localeCompare (TL;DR: the defaults are quite decent):

function circularShift(line) {
    let [head, ...rest] = line;
    return [...rest, head];
}

function circularShifts(line) {
    let all = [line];
    let last = line;
    for (let i = 1; i < line.length; i++) {
        last = circularShift(last);
        all.push(last);
    }
    return all;
}

function lineComparator(l1, l2) {
    let first = l1.join(" ");
    let second = l2.join(" ");
    return first.localeCompare(second);
}

/**
 * @param {string[][]} lines
 */
function kwicIndex(lines) {
    return lines
        .map(line => circularShifts(line))
        .flat()
        .sort(lineComparator);
}

Under the assumption Parnas' estimate of the required effort was not hyperbolic, and let's take the lower end of it (a 40h work week), that results in a factor of 120 in improved productivity (120 * 20min = 40h), simply for having bog-standard Javascript available (I presume it will translate to the same order of magnitude in most higher programming languages), and all without having to bother myself with a Large Looting Model.

Why did you start blogging in the first place?

The idea was brewing in the back of my mind for quite some time. What I had online before I wrote my first post here is best described as a dumping ground for a few slide decks of talks I gave at either some unconference or a local web developer meetup, nothing around it what would otherwise resemble a personal website. Just like the rest of the world I stopped attending in-person events during the early 2020s. I had also collected material in a personal wiki over the years, for my reference and as memory aid mostly, but I considered nothing of it as good enough to be worth putting it online. This left me with the nagging thought, that my notes would deserve a bit of polish. In early 2021 is stumbled over a post from Mike Crittenden: Prove you can write regularly before you think about the tools. In it he gave a great piece of advice: You’re not allowed to care about the tools until you’ve proven you are capable of regular writing., and he had embedded Rakhim's Blogging vs. blog setups comic to illustrate his point. I remember that I nearly spilled my coffee over the "weird dude who writes raw HTML". It captures quite well how many folks, including me, rolled websites in the early 2000s. So, circumstances aligned, I thought, tools be damned, I can put something online just as I used to do 20 years prior, sat down, opened VS Code, wrote a few lines of text and a few lines of styles, pushed it to my github pages and said to myself "Let's see how long the impulse will last". It is lasting for more than four years now.

What platform are you using to manage your blog, and why do you use it?

I'm not sure if platform is the right term, but the site runs on the most under-powered (and cheapest) VPS I was able to find. The operating system of the server is Ubuntu, which I simply choose out of familiarity. On it I use nginx to serve the static content and reverse-proxy half a dozen nodejs applications that support minor parts of the website, like the random-page redirect and the webmention receiver. The site itself is built with a static site generator that I tailored to my needs (or should I rather say idiosyncracies?). I deploy it simply by copying files via ssh to the server.

Have you blogged on other platforms before?

I started out on Github pages for deployment of content. But to create the site itself, I always rolled my own to some degree: from writing everything manually, over gradually scripting away some of the repetitive parts (like feed generation), to the current version. It seems I am habitually unable (or perhaps unwilling) to accomodate to other peoples ideas what blogging ought to look like.

How do you write your posts?

Many articles still start in my wiki (a TiddlyWiki instance). Once they've reached a certain point of maturity, I copy them over to a file in VS Code, where I mark them up properly. There are some sections of the site that I edit directly in VS Code, for example the rather recently introduced notes or the now page. For final editing I tend to use a screen-reader on the generated page, because I often become blind and don't see duplicated words or sentences that miss a verb. Hearing the text read out loud by another voice helps with that.

When do you feel most inspired to write?

Inspiration is a perishable good, and it comes rarely when I actually have time and energy to actually sit down and write. What helps is a good chunk of uninterrupted time (a rather rare good), being well-rested, or at least well caffeinated. Most of that time is already allocated to non-negotiable duties. What ever I can scrape away in the evening or at the weekend is when I work on shaping muddy thinking into at least halfway coherent sentences and paragraphs.

Do you publish immediately after writing, or do you let it simmer a bit as a draft?

It depends, the notes are one-shot mostly, articles rarely. I usually need multiple sittings for one and for editing most often I need an extra session after the writing. When I have a particularly good day, an article might be written and edited in the same session, but that has not happened very often.

What is your favourite post on your blog?

Always the next one of course ;)...

I will read it sparked quite a few connections to and some really great conversations with fellow bloggers. Other than that, I think the write up on my webmention receiver implementation was decent, and I like the literate programming style easter egg.

Any future plans for the blog?

Many ideas, but very few plans. I'm mostly happy with the setup, the only technical aspect I'm actively thinking about is offering more differentiated feeds, for example per topic / category. Apart from that I want to make more use of the system I have.

Who will participate next?

I guess who ever feels like it? But I'll pass the virtual baton over to Dario.

Bill Beaty, in over 30 years of running Science Hobbyist, has created lot of content. Answering the question how could create so much he writes: I live on the internet, in an ancient unix shell account [..]. If I should ever type up a little textfile during other activities on the computer, it only takes me ten seconds to put it on my webpage. Unlike most people, for me the barriers against publishing on internet have entirely evaporated. Such a way of working is of course a trade-off. As websites grow over time, usually at one point one may want to change some part common to all pages, like a header, a footer, the side navigation, which is an easy task if you use a content management system, not too hard with a static site generator, and, if everything is rolled by hand, probably comes with so much friction that in the end you would not even bother to try.

But tooling itself is also not a "free lunch": you have to keep up with it, and tools can change in undesirable ways. It might lock you in, so that swapping it out also can present a major cause of friction. My friend Lars-Christian recently decided he had enough of Wordpress, so first he wrote his own static site generator, and then, as he understandably did not want to pay a premium for Wordpress hosting anymore, had to move his blog to a new host. Once more, a such decisions exemplify trading one kind of friction for another one. Linus Lee argues that tools that parallel our minds can multiply our creativity and productivity, by removing the invisible friction of translating between our mental models and the models around which the tools are built.

Next, take Rakhim, who wanted to revive his website. In his own words, he just wanted to write again. But his tools then again were a cause of friction: after installing a new version of Hugo things broke. After 25 minutes of attempts to fix it I though "yeah, that's it". Now, in what I can with the utmost admiration only call a proper yak-shave, instead of abandoning the thought of writing he decided to building out a blogging platform to his own ideals. But, it would go without saying, one is not done dealing with friction once a platform is built, as shows the example of Herman Martinus, the creator of bearblog. When the free tier he offers started to attract too many spammers and SEO farmers (no good deed goes unpunished...), he had an ingenious idea to put friction to good use to deal with them and introduced The Frustration Loop.

Friction in the non-metaphorical sense is a natural phenomenon, and - as any force of nature - indifferent about societal or moral categories. It applies to all, for good or for bad. In the context of publishing on the web it is no different. The orders of magnitude difference between the billions of people on silos compared to the maybe some ten thousands on the independent web is a reminder that of one of the core principles of the indieweb, that UX and design is more important than protocols, formats, data models, schema is both valid and collectively neglected by the community. The silo platforms, even if their driving motives are of a mere pecuniary nature, demonstrate that simpler ways are feasible. I consider this type of friction to be similar to what software engineers, after a notion introduced in Fred Brooks' seminal essay No Silver Bullet, call the accidental complexity of a system:

[T]o see what rate of progress we can expect in software technology, let us examine its difficulties. Following Aristotle, I divide them into essence — the difficulties inherent in the nature of the software — and accidents — those difficulties that today attend its production but that are not inherent.

Just as Brooks notes that advances can do no more than to remove all the accidental difficulties from the expression of the design. The complexity of the design itself is essential, so is the essence of publishing on the web not the tools that are needed to build and serve a stream of bytes to be rendered in the browser. The essence are the ideas itself. The authors of the classics of literature, the influential thinkers, philosophers and scientists did not concern themself with printing, typesetting or bookbinding. They may or may not have held these crafts in high regard, ultimately they were inessential to their work.

If the accidental complexity of publishing on the web were resolved for good, we would be, in the words of William Zinsser, still stuck with the same old job of saying something that other people will want to read.

The indieweb carnival for June 2024 is hosted by Andrei, who has dealt out the topic DIY — Something from (Almost) Nothing, which I think is a brilliant prompt. I'll use this as an opportunity (or excuse?) to create something that was on my to-make list for quite some time now: adding a webmention receiver endpoint to my site. I think as this also ticks a few boxes of list principles of building on the indie web: Make what you need, use what you make, document your stuff, open source your stuff and have fun.

To additionally honor the motto "doing something from almost nothing": I've chosen to do it in plain JS and restrict myself to the APIs that are in the current LTS version of Node.js, no third-party libraries, no npm install, just the platform.

So, let's dig out the specification, read through what is expected if you want to receive webmentions and start programming.

Upon receipt of a POST request containing the source and target parameters, the receiver SHOULD verify the parameters [..] and then SHOULD queue and process the request asynchronously, to prevent DoS attacks. [..] If the receiver processes the request asynchronously but does not return a status URL, the receiver MUST reply with an HTTP 202 Accepted response.

So, the endpoint expects to receive POST requests, with a form-encoded payload. It has to do some sanity checks synchronously (validation of the request), and then in a seperate process, do some deeper inspections to ensure that the webmention, well, actually does mention a URL of the receiving server, so:

const server = http.createServer((request, response) => {
    let content = getBody(request);

    content.then((body) => {
        let mention = toMention(body);

        if (isValid(request, mention)) {
            if (isEnqueueable(mention)) {
                queue.push(mention);
                response.writeHead(202);
            } else {
                response.writeHead(500);
            }
        } else {
            response.writeHead(400);
        }
        response.end();
    }).catch(() => {
        response.writeHead(500);
        response.end();
    });
});
server.listen(PORT);

The platform-features-only policy already has implications. The first one: reading out the body with the callback-style API of nodes http.IncomingMessage is a bit inconvenient, that's why I wrap it into a little helper:

async function getBody(request) {
    return new Promise((resolve) => {
        let body = [];
        request.on("data", part => {
            body.push(part);
        });

        request.on("end", () => {
            let result = Buffer.concat(body).toString();
            resolve(result);
        });
    });
}

As the receiver is essentially a single endpoint, I don't really bother with proper routing. Still, had I opted into using a framework, say the venerable express, some of the sanity checks, that I now need to push down into the request validation logic, would have implicitly been handled (for example: I would not need to check whether I've indeed received a POST request).

function isValid(request, mention) {
    let isBadRequest =
        BAD_REQUEST_PREDICATES.some(isBad =>
            isBad({ mention, request })
        );

    return !isBadRequest;
}

The request validation is straightforward. I check against a number of reasons if the request should be considered to be a client-side error, and if nothing is found, well then it is at least worthy of some further processing.

The validation is called with the request and a webmention object that is extracted from the request by wrapping the form-encoded payload into URLSearchParams, and looking for the expected properties:

function toMention(body) {
    let payload = new URLSearchParams(body);

    if (!payload.get("source") || !payload.get("target")) {
        return null;
    }

    return {
        source: payload.get("source"),
        target: payload.get("target"),
    }
}

Now, what indicates a client-side error? Well, not passing a few basic sanity-checks, each a one-liner:

let BAD_REQUEST_PREDICATES = [
    ({ request }) => request.method !== "POST",
    ({ request }) => !request.headers,
    ({ request }) =>
        request.headers["content-type"] 
            !== "application/x-www-form-urlencoded",

    ({ mention }) => !mention,

Further according to the spec: The receiver MUST reject the request if the source URL is the same as the target URL.

    ({ mention }) => mention.target === mention.source,

The receiver MUST check that source and target are valid URLs and are of schemes that are supported by the receiver. (Most commonly this means checking that the source and target schemes are http or https).

    ({ mention }) => !URL.canParse(mention.source),
    ({ mention }) => !URL.canParse(mention.target),
    ({ mention }) =>
        !(mention.target.startsWith("https://")
            || mention.target.startsWith("http://"))

Thank goodness URL.canParse is in the ES2022 baseline - otherwise you would need to wrap the URL constructor in a try catch block to check whether it is actually parsable. Eww...

The receiver SHOULD check that target is a valid resource for which it can accept Webmentions. This check SHOULD happen synchronously to reject invalid Webmentions before more in-depth verification begins. What a "valid resource" means is up to the receiver.

    ({ mention }) =>
        ALLOWED_TARGET_BASE_URLS.length > 0
        && !ALLOWED_TARGET_BASE_URLS.some(
            allowedBaseUrl => mention.target.startsWith(allowedBaseUrl)
        ),

The allowed targets are a list of base url, that is read in from a file when the server is started. In another iteration, that file could be filled with a sitemap.xml (or something similar) and do a somewhat better check than just a startsWith, but for the moment, I consider it good enough for my purposes.

So a first round of sanity checks are passed. Before I proceed and try to verify the webmention, I do some more synchronous checks, to prevent potential abuse, that are not in the spec: first of all, a mention that is enqueued for processing already will not be added again and also legitimate-looking mentions that failed to verify are stored and after half a dozen attempts will get dropped, as I don't want my machine to become part of a DDoS attack on somebody elses server. Lastly: I will re-verify a mention earliest after 24 hours have passed.

function isDuplicate(mention) {
    return function (existing) {
        return existing.source === mention.source
            && existing.target === mention.target;
    }
}

function isEnqueueable(mention) {
    let alreadyEnqueued = queue.find(isDuplicate(mention));
    if (alreadyEnqueued) {
        return false;
    }

    let failedValidations = failures[mention.source];

    if (failedValidations && failedValidations > 5) {
        return false;
    }

    let alreadyValidated =
        received.webmentions.find(isDuplicate(mention));

    if (alreadyValidated) {
        let elapsedTime = Date.now() - alreadyValidated.validatedAt;
        if (elapsedTime < (24 * 60 * 60 * 1000)) {
            return false;
        }
    }

    return true;
}

Ok, with this the actual endpoint is done, now for the verification part of the webmention, which is done in an async job:

If the receiver is going to use the Webmention in some way, (displaying it as a comment on a post, incrementing a "like" counter, notifying the author of a post), then it MUST perform an HTTP GET request on source, following any HTTP redirects (and SHOULD limit the number of redirects it follows) to confirm that it actually mentions the target.

I haven't really made up my mind how I will use them, but I think I will want to use the webmentions at one point in time, so I should at least superficially try to verify them.

The receiver SHOULD use per-media-type rules to determine whether the source document mentions the target URL. For example, in an HTML5 document, the receiver should look for <a href="proxy.php?url=https%3A%2F%2Fholzer.online%2F%2A">, <img href="proxy.php?url=https%3A%2F%2Fholzer.online%2F%2A">, <video src="proxy.php?url=https%3A%2F%2Fholzer.online%2F%2A"> and other similar links. [..] If the document is plain text, the receiver should look for the URL by searching for the string. Other content types may be handled at the implementer's discretion. The source document MUST have an exact match of the target URL provided in order for it to be considered a valid Webmention.

Ok, I am cheating here... My go-to library for server-side DOM processing would be JSDOM, which I also use in my static site generator, but as I've constrained myself to the standard library I'll do only the simple check on whether the target URL is present anywhere in the source. It's a first iteration after all.

async function validateMention() {
    if (!queue.length) {
        return;
    }

    let mention = queue.shift();

    let response =
        await fetch(mention.source)
            .catch((e) => console.error(e));

    if (response?.ok) {
        const data = await response.text();

        // only a heuristic - prone to false-postives!
        let isMentioned = data?.includes(mention.target);

        let existing = received.webmentions
            .find(isDuplicate(mention));

        if (existing) {
            existing.validatedAt = Date.now()
            existing.isMentioned = isMentioned;
        } else {
            received.webmentions.push({
                ...mention,
                validated: true,
                mentioned: isMentioned,
                validatedAt: Date.now()
            });
        }

        await writeFile(
            path.resolve(__dirname, mentionsFile),
            JSON.stringify(received)
        );
    } else if (response?.status === 410) {
        let existing = received.webmentions
            .find(isDuplicate(mention));

        if (existing) {
            existing.validatedAt = Date.now()
            existing.deleted = true;
        } else {
            received.webmentions.push({
                ...mention,
                validated: true,
                deleted: true,
                validatedAt: Date.now()
            });
        }

        await writeFile(
            path.resolve(__dirname, mentionsFile),
            JSON.stringify(received)
        );
    } else {
        if (!failures[mention.source]) {
            failures[mention.source] = 1;
        } else {
            failures[mention.source]++;
        }

        await writeFile(
            path.resolve(__dirname, failureFile),
            JSON.stringify(failures)
        );
    }
}

setInterval(() => {
    try { validateMention() } catch { }
}, 500);

I've omitted what little boilerplate there is, e.g. for reading out the command line arguments, the essence clocks in short of 200 lines, which you can look at in their entirety in the repo on Github.

I could end here, making this another instance of the cliché "developer writing a blog about a how he developed his blog", but I'd also like to do add some more thoughts (or navel-gazing - I let you be the judge of it) on what it means to put something out on the open web as an independent person (and not on somebody elses platform).

In an article that I found via Wouter Groeneveld a fellow webmaster going by the handle of @gar0u bemoans:

IndieWeb is a social club for developers, and apparently, not for me.

[..]

When some full-stack developers struggle to have WebMention implemented on their blogs, I learn the answer. It’s not child play for the unenlightened.

And there is undeniable some merit to that observation. Take the code I wrote: quite a chunk of was designed to deal with potential abuse (as already forseen by the wise folks who wrote the spec). I mean, it is just a fact of life, nobody who writes web apps for a living will lose too much sleep about it - but what does it tell the ambitious amateur (in the most positive sense of the word: those who operate a personal site for the love of it), when the implementation of a technology for interpersonal connection concerns itself primarily with hostilities of unknown thirds in mind?

And then, via Matthias Otts newsletter series Own your web, I found an article by Sebastian Greger, which sheds quite some light on the legal implications of web mentions. Well, I recently paid the annual fees for my insurance against lawsuits, but still, I can imagine how such stuff would deter many folks from participation, even if they jumped the technological hurdle. I know it would have detered younger me.

A pioneer of the computing field, Bob Barton, once said that System programmers are the high priests of a low cult. Maybe we have climbed one or two little rungs on the long ladder of abstractions, but that pithy statement, made in 1967, still feels quite valid. Staying in that metaphor, I want to end with a more optimistic description of the capabilities of this still very young medium, that we are shaping together: Dave Rupert recently called it A holy communion:

You and I are partaking in something magical.[..]

Our thoughts, words, and images transfigured through time and space as ones and zeroes. Circumnavigating the globe in milliseconds. An immaculate relay. A holy exchange.

[..] We would do well to preserve its sanctity.

Indeed, therefore — Ite. Missa est.

The Pew research center recently published an estimation that 38% of webpages from 2013 are no longer accessible. And, as if to underline the point, a few days ago the Internet Archive was hit by a DDoS attack. Preserving the web continues to be an open problem, and this is a reminder to archive things you care about locally, where it is possible. Given that state of affairs, here's a few articles I found interesting, which against the odds are still online. They share a common thread: all of them are about exemplary software design ideas.

Asher Olsen lays out the internals of the First Lisp Compiler. What I find fascinating is how many high-level features could be realized by ingeniously mapping some syntactical sugar over lambda calculus to the instruction set of a computer from the mid-1950s. However, I've to admit that the details still go over my head, but what else is to be expected by the language from which the gods wrought the universe (at least ostensibly so, as xkcd reminds us)?

So let's come back to a lisp more suitable for mere mortals: Javascript. A few month before Musk decided to cure my addiction to the bird site (and contributing a bit to the 38% of URLs that have vanished), Frank Force tweeted a program, fitting in 256 bytes of JS and HTML, which renders a city landscape using ray casting. Daniel Darabos shortly afterwards reverse-engineered and dissected its inner workings in Decoding A City In A Bottle. Frank recently also wrote a break down of his code.

Last, but not least: Greg Wilson has made his book Software Design by Example - a tool-based introduction with JavaScript freely available only. I've still got to work through some of the chapters, but it is the kind of book that would be on my shortlist for a Great books of software engineering curriculum. If you're into making software, you might also want to checkout the rest of Gregs impressive bibliography.

As I reviewed my bookmarks from the weeks 20 and 21 of 2022, I noticed that a recurring motive were neat old ideas.

First I found an article by Mark Nelson about Operator precedence by textual substitution, which describes a suprisingly simple method to handle the evaluation of arithmetic expressions, that stems from the early 1960s. I once had an job interview, where I was asked about this very problem. I went down with the "convert infix to reverse-polish-notation and traverse the tree" path. I wish I had known this neat little trick then...

Tom Stuart shows in Programming with Nothing how you can take nearly everything away from a programming environment and still accomplish computations. In restrospect, this is what I would have needed in the theoretical computer science course that was my introduction to the lamdba calculus.

And as a contrast, on the other end of the spectrum, George Lungu, on his blog Excel Unusual shares cool ways of solving engineering and science problems and modeling and visualizing natural phenomena. Hundreds of spreadsheats with some very deep ideas available. Also a good reminder to never underestimate spreadsheats.

Lastly, if you are into well-hung (or battle-tested) ideas, you might enjoy the archives of Lindy Hacker News, which collects classic articles that get frequently reposted on Hacker News.

So, half a year has past since I wrote the last weeknote. This is arguably an indicator that the a fixed schedule does not quite work out for me. On the other hand, the first reboot of week-noting was after stopping for a complete year. So, MTTR (mean time to recovery) is clearly improving.

Some time ago I wrote about a little bookmarking app, that I made for myself. Well, I tweaked it a bit since, and now it can group my link collection by various criteria, one of them being the weeks that I added the bookmark. And so I thought, let's revisit what is in there and share a few articles that stood the test of time. So here's half a dozend, that stem from week 19 of 2022 (that is when I found them, which is not necessarily the same week or even year that they were written), that I think you might enjoy.

Justin Etheredge discusses the trade-offs between RDBMSes and NoSQL, with a focus on the merits of the former. If you are in a position to give advice to software engineers early in their career, consider telling them that Relational Databases aren't dinosaurs, they're sharks.

Maybe it is no conincidence that I also bookmarked Kellan Elliott-McCrea list of Questions for a new technology in the same week.

Allen Downey has published over a dozend textbooks books on computer and data science topics. He generously makes them available for free under Green Tea Press, print versions of many of his books are also published via O'Reilly and No Starch.

Andrew Healeys post about on A Tiny Project, From Inception to Deployment is in my eyes examplary with regard to many aspects: the chosen scope, balance of explanation, visualization and showing working code and the overall length. Frankly that is the kind of writing on software I aspire to do more often, as I am often struggling with that very balance. Either I become lost in line-of-code-level minutiae or am rambling in the vague, most often ending with an article being canned, or lingering forever in the limbo that is my drafts folder...

John Hoare has a word of advice for those of us who maintain personal websites: Design for Creation. The gist, for those in a hurry: a good design helps you write and publish. And a bad design is one which gets in the way, and makes sharing your ideas difficult.

I've intentionally put that advice prior to sending you down the rabbit holes that Wouter Groeneveld has collected in Cool Things People Do With Their Blogs. Consider yourself warned ;)

And a last one, that is also from the links of week 19/2022, but as I don't hot-link image resources on general principle, I've put is as a copy on my site. I use memes rather sparingly, but this one is pure gold that I've sent to quite a few of my coworkers at appropriate points in time:

In an age of silos and centralization, trying to curate the web may seem like a folly. Luckily, if you're looking, you'll still be able to find plenty of individual initiatives, that provide lenses into the worthwhile parts of the web. If you have a few hours to spare to go down rabbit holes, I'd heartily recommend Toms bukmark.club, a little directory of little directories.

Such directories make me reminiscent of what Yahoo or DMOZ represented in their hey-days: portals to the knowledge of the world and options to connect with people over shared interest, that would not have been thinkable just a few short years prior. They are not the only form of curation and sharing links though: from the venerable blogroll, over regularly updated link blogs, markdown files in a git repo (like the various awesome-lists), OPML exports of feed lists - there are many little acts of connection, that always were the heart and soul of the web.

Yet, in its vastness I perceive all those attempts of bringing order as far scattered. Now, one could just shrug and say that such is the nature of the independent open web, but I think there is a ton of potential to unearthed. How can the good things be amplified in a time where we are collectively being drowned in automatically generated noise?

Clay Shirky in his 2005 essay Ontology is Overrated layed out a critique of the conceptual shortcomings of large centralized web directories, and reasons why Google ate Yahoo's lunch. Among many observations he wrote Browse versus search is a radical increase in the trust we put in link infrastructure. It is congruent with my memory - but two decades down the road, that initial increase in trust has eroded, if not completely evaporated. The Big Tech corporations all have shown their true colors rather fast, and all tech-optimism aside, noone should not be suprised that corporation first and foremost want to make profit.

So how could the disjoint efforts of idealistic individuals be brought together, without introducing centralization or putting any burden on those who already do the work of curation itself, maybe even in a form where the sum is more than the parts? To that end, I'd like to explore an idea, that Shirky formulated, a bit more in depth:

You don't merge tagging schemes at the category level and then see what the contents are. [...] You merge from the URLs, and then try and derive something about the categorization from there.

Suppose, you would like to make use of a number of smaller link collections: manually browsing through every single link collection known to you would likely turn out to be a tad cumbersome. What if you could peruse a personalized aggregation of collections in a single place? Could you somehow build a set-union of a (hand picked) selection of web directories?

To paraphrase Shirkys insight: As much as the size, the organization structure, categorization schemas, and formats of a collection or directory may vary, there is a lowest common denominator: each collection could be modelled as a flat list of entries, where each entry represents link. The humbe URL pulls double duty: it is both the most important bit of information (i.e. arguably would represent the essence of an entry all by itself), as well as the ID and, taking the plurality of collections in existence into account, also would act as the foreign key. With regard to the metadata, certainly a few more properties on each items are desirable. I think a title is rather uncontroversial, also a short description might be be a common characteristic. And as many directories are not just lists, but also have some sort of categorization schema, that should be represented somehow as well. Here it starts to get less clear. Are categories disjoint? Can one entry be part of multiple categories? Are there multiple levels of hierarchy? Are tags used additionally? Can categories be thought of as a special case of a set of tags?

Having arrived at a point where such ambiguities might lead to analysis paralysis I need to invoke Ableson's dictum: people who believe that you design everything before you implement it, basically are people who haven't designed very many things. So, let me start sketching out the problem in code instead of words.

class Entry {
  constructor(url, title, tags) {
    this.url = url;
    this.title = title || url;
    this.titles = [this.title];
    this.tags = [...new Set( ...(tags || [])];
  }
}

Ok, not much more than an anemic model, let's doodle a bit with it: A single entry in a web directory has a URL, a title (and where that is forgone the URL can be asked to pull triple duty) and associated tags (just a deduplicated list of strings). If there were two directories that want to compare notes, an entry could be enriched by adding the the metadata from the other source to its properties:

  enrich(entry) {
    if (this.url != entry.url) { 
      // basic sanity check.
      // no use to compare apples and oranges
      return;
    }

    if (!this.titles.includes(entry.title)) {
      this.titles.push(entry.title);
    }

    this.tags = [...new Set([...this.tags, ...entry.tags])];
  }

From another angle: a single collection, a web directory for example, could be represented as a map of URLs and associated metadata, that are tended to over time, by adding entries one by one:

class WebDirectory {
  #knownUrls = new Map();

  [Symbol.iterator]() {
    return this.#knownUrls[Symbol.iterator]()
  };

  add(entry) {
    if (this.#knownUrls.has(entry.url)) {
      this.#knownUrls.get(entry.url).enrich(entry);
    } else {
      this.#knownUrls.set(entry.url, entry);
    }
  }
}

Now two directories could, as Shirky suggested, indeed be merged from the URLs.

  join(otherDirectory) {
    for (let [url, entry] of otherDirectory) {
      this.add(entry);
    }
  }

That little model by itself doesn't yet achieve very much. There are two important, but uncovered aspects for this to become actually useful: a way to map the structures of existing collections and directories into such a common format and then on top of these mappings there need to be one (or rather: many) views.

Oh dear, I think I've just wrote myself a high-level spec for a side project...

I am using nginx as reverse proxy for a handful of small applications. I recently added one where for convenience I only wanted to require basic authentication for the endpoints that write data.

The nginx configuration files are not exactly my daily concern, so to save future me (or maybe future you) some headaches with trial and error, here is what worked (and what didn't).

As I wanted to only secure POST requests, my first attempt was to formulate it using positive logic with an if directive. Didn't work out and the good folks at nginx even wrote an article about why if is evil when used in location context - which happens to be exactly the context I was in.

The alternative is to enumerate HTTP methods as being exempt from authentication in a limit_except directive. If you model your API like a halfway sane person (and not for instance use GraphQL...) that means GET (HEAD and OPTIONS could be thrown in for good measure). But beware: the proxy_pass directive must not be put into the limit_except block. It will make the nginx config invalid. nginx -t will yell at you, and if you restart anyway, or like a certain person don't bother to testing the config changes before you restart, the server will crash immediately.

So, the final snippet looks like this:

    location /my-application/api/ {
        limit_except GET {
            auth_basic "Restricted";
            auth_basic_user_file /path/to/.htpasswd;
        }

        proxy_pass http://127.0.0.1:12345/;
        proxy_buffering off;
        proxy_set_header Host https://$host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }

Manuel host this months indieweb carnival on the topic of Digital Relationships. He also, nearly one year ago, happens to have started an initiative in which among several others I have followed suit in: an offer to become regular reader of your personal website. I'll take this as an opportunity to reflect on relationships that were initiated through this and the potential of the digital medium to create genuine personal connections.

While some soulless traffic metrics might indicate otherwise, the independent, non-commercial personal web is what I consider to be the most important and exciting part of the web. One of my favorite Alan Kay quotes goes along the lines that having perspective is worth 80 IQ points. And I'd be hard pressed to come up with many better ways to gain a broader perspective, than through a well-curated feed reader. Of course, curation of and, as a prerequisite, discovery of people to read are important ingredients that still are in need of more sustainable solutions than we have currently. The personal web sadly is being drowned in a sea of commercial noise with orders of magnitude bigger volume, which is induced by the major search engines and the digital dopamine slot machines (which I neither seriously nor ironically want to call social networks) that attempt to sell our eyeballs to the highest bidder.

Henrik Karlsson called blog posts complex search queries to make fascinating people route interesting stuff to your inbox. And I can attest that there is truth to that. Joining the "readers club" was the most successful query I wrote so far. I am certain that nothing I would have entered into BingoogleGPT would have sent me to Rodrigos tech journalism, Yaidels points of view, Kimberlys commonplace book, or Stephanies takes on popular culture. I would also have missed out on many delightful conversations that were initiated as a reaction to something written in public - and I'd be poorer for it. These connections that were initiated through the simple act of reading blog posts and writing a mail are something that I cherrish.

And when I look at the overall state of the personal and independent web, I am reminded of a pre-digital phenomenon that is known as the "republic of letters" (a term, mind you, coined by those who participated!). In the 17th and 18th century philosophers, scholars, university teachers, mediated by the regular exchange of personal correspondence, formed an intellectual long-distance community all over Europe. I find it remarkable that while the lands overwhemingly where governed by absolutist monarchs, which only knew of subjects, the intellectuals of their age considered themselves citizens of a republic. I can't help but to draw a parallel: On the dominating platforms of today, nobody will ever be a citizen, not even a customer. We're only good enough to be a user. But outside of the walled gardens, there are still ample opportunities. We can form relationships, participate in debate, cultivate a corner of and shape the digital medium. All these verbs - nota bene - have in common that they are active.

To conclude: the personal web is designed and destined to be a two-way street. Being part of it is a choice I would love to encourage many more people to make. Long may flourish the republic of mails.

As my family and I are looking forward to move into a new home in January, the next weeks will be filled by throwing out junk that accumulated over the years, packing up our belongings into boxes, looking for new furniture etc.pp. and I'll likely be hard-pressed for time to write — therefore, as a short bleep before I go radio silent, I want to cluster some ideas that I'd like to pick up, once I have a little more time at my disposal again.

• I'd like to finish a draft about the implementation of the little web component that I mentioned previously. This one has the strongest chance to still get done this year.
• There's a pile of scattered notes on microfrontend architectures, that have accumulated over the nearly eight years that I'm working on that topic professionally. I'd like to consolidate them with my personal experience and opinions on it. As that might get out of hand and maybe deserves a longer treatment, I'm thinking about experimenting with some kind of preplanned article series.
• Earlier this year I mused about the great books of software engineering, but the article was a bit unconclusive. I will have to pack up my library soon and thought, when I unpack the books, I'd like to do some reviews and summaries on a selected few items.
• Then, and this one is a bit of a moonshot topic, earlier this year, I took some interest in the current state of rapid application development. I'm still occasionally asking myself why, a quater of a century after the final release of Visual Basic 6, nothing has emerged that lives up to its developer experience, when it comes to making UIs and wrapping them up with a datasource. I have a some hypotheses about that, though...
• The quality of the major search engines have deteriorated over the last years to the point where their results are basically garbage. The rise of LLMs has, if anything, accelerated their enshittification. Plus ça change, plus c'est la même chose... I'd like to dive into the topic of curated web directories and explore whether there are lessons to be learned from the demise of Yahoo and DMOZ and if smaller directories could somehow become (again) a backbone for finding information on the web.

If any of you'd like to talk about such stuff, feel free to shoot me a mail. Until next time, take good care :).

I've recently made a few accessibility improvements for the site. For one, I finally got around to add a skip-to-content link to the navigation, mostly following along an article from CSS Tricks, with very few tweaks. And I replaced the sorttable script for the tables of content with a custom-element I wrote, in order to make the table columns also sortable via keyboard navigation. On the same note: some time ago I wrote that small things matter for accessibility, which as a general principle is true, but I recently learned that the example which triggered the article is not correct anymore, because bold and italics actually aren't read by screen readers after all. That was a feature of the NVDA screen reader, but not a very popular one, therefore it got disabled by default. Generally there is no audible distinction between i/em or b/strong tags for screen reader users.

Also three weeks ago, I had the pleasure to participate in my first post-pandemic in-person developer event (which incidentally has been my last pre-pandemic conference as well...), the unKonf in Mannheim. The unKonf is basically a barcamp, but it is traditionally opened with a keynote, which this year was given by Uwe Friedrichsen on the topic of sustainability patterns for IT. One fact that suprised me was that the global IT landscape causes more greenhouse gas emissions than the traffic of the worldwide logistics industry. I can only recommend the slide deck, which is filled with tons of interesting and important pointers. One take-away was that measuring and visualizing resource and energy consumption of a system is a necessary first step to move towards a more sustainable IT. Apart from that, there were many other great sessions and the "hallway track" made me realize how much I actually have missed to talk shop (outside of my actual shop).

Since this week my parental part-time is over, I'm back to working my standard 40 hours. With so much of my time to available to the company again, sure enough somebody deemed it a good idea to assign a training course on GDPR to me. Ok, in all fairness, that course was probably assigned to just about everybody in the department, in order to tick a box on some spreadsheet, for you know, a bit of corporate cover your ass can go a long way sometimes, but I'm digressing... Anyway as a good compliant employee, I started the course, hoping for the best. It turned out to be gamified, in - I kid you not - the form of detective's story and the look and feel and a UX of a badly programmed game, that reminded me very much on the numerous hobbyist game from the hey-days of Flash...

But on to a more serious matter of user and developer experience. My son is currently digging a bit into programming with Scratch on the Raspberry Pi. Following along a tutorial, after making the figure on the screen do some funny things, tweaking and exploring here and there, he got to the point where he had a little game that he wanted to spice up by checking for a game over condition.

So, a quick test: the player has lost their last life, tell me, why is the if branch not executed?

The logic looks quite sound, doesn't it? Perhaps a different notation helps to spot the problem faster:

lives--;
if ("lives" < 1) {
    console.log("Game over");
}

So, what happened? While following the tutorial he didn't drag the variable as a block, but typed it out into the field, which makes it a string literal (a legitimate thing to compare something with). Referencing the actual variable looks like that (not quite an optical difference like night and day):

As block-based programming languages have found their way into school curricula as well, I was wondering how many kids' enthusiasm for computing might get curbed by such little matters of affordances that cause confusion or frustration.

Last, but not least, I added two items to the site: a client-side full-text search (powered by lunr.js) and a Colophon, because I always like to read about the website setup of other people for inspiration and also to give credit where credit is due to a few libraries I use, which were created by people who generously open-sourced them.

So, after I've managed last week to switch over to my custom SSG for building the website, this week I spent time to play around and explore a few things, that I would not have done, if I needed to maintain them manually.

The first addition, inspired by Daniel Fedorin, is a content graph, which visualizes how connected my posts actually are. As it turned out, not that much and even less than I had thought. A lot of articles are "orphaned" (meaning, not linked to internally). The second, less pretty, but probably more useful, outcome are automatically created backlinks for each article. The last little new thing is syntax highlighting with Prism, which yielded the philosophical question: does my css still count as classless, if the classes in the markup are only created programatically?

To sum up, what started out as the musing of a "weird dude who writes raw HTML", is moving up on the x-axis of this chart towards the static gen basin. For as, much I would like to pretend otherwise, tools matter at least a little bit.

For those who read my site via the feed, this is also the reason why some old articles might have popped in your reader up all at once. The old script was heavily depending on a certain regular structure of the directory, and I would have needed to run the rss generation script for each category and then somehow properly merge the resulting xml files, which I found to be too tedious, so the weeknote category of the site was left out (which was another driver for the rewrite).

There is a great xkcd about how much time you may spend on automation before yielding negative returns. If all the tedious stuff took about 5 minutes per post and I'll write once a week, about 21 hours of programming amortize themselves over the course of five years. But even if programming a custom SSG turns out to be uneconomically from the point of timesaving, it still was fun, which is enough reason to do it. I've put both the the generator, which I named intertwingle, as well as the source of my website on github.

On to a completely different subject. This week at work, I heard that upper management is looking into establishing tools that create metrics from our repositories and issue tracker to be reported to them. I don't know whether it is merely a conincidence that a month or so ago an article about measuring developer productivity by a big management consultancy made the rounds. Kent Becks response in two articles (Part 1 and Part 2) is quite good. In a debate on Linkedin Kent also delivered a soundbite, that sums it up up nicely: McK is offering false certainty to replace honest ambiguity. Anyway, I think it is rather obvious why advice on software engineering from a company that mainly sells political capital and powerpoint presentations (even if the latter is Turing-complete...) needs to be taken with more than just a grain of salt.

And as I was in digital clean-up mood, I also pruned my wiki. Consolidating notes and throwing away drafts that didn't lead anywhere left it both over a hundred entries smaller and more useful to me than before. Sometimes less is more.

At home I don't have so much uninterupted leisure time (emphasis on uninterupted) available, which means for most books of my choice I need quite some time to get through. But last week I managed to read Rolf Dobellis Die Kunst des klugen Handelns ("The art of acting wisely" - don't know if an english translation is published) within three days. It is about cognitive biases and errors in thinking and consists of mercifully short chapters, which fitted quite neatly into many fractured timeslots, that would otherwise have been rather dead.

Also I'm really starting to feel jaded by Substack based "blog-letters" (or how shall one call nominally newsletters with fully accessible public archives?). I hate it when the reading flow is interrupted by their annoying overlay, which is added very gradually while scrolling down, so that while you read, you know that if you want to continue, you'll have to dismiss the same damned dialog all over again in a few seconds. It totally ruptures concentration and flow. Yeah, I know, creators want to capture their audience, and free is hardly ever really free, we all have to pay our bills, but this particualar interruption in the middle of articles starts to feel for me like being hit with a hammer. There is ample evidence that forcing the brain to make decisions leads to cognitive fatigue and depletes the abilities for self-control. I'm picking on Substack here, because they are currently very popular in my online bubble, but the same could be said about many other websites and platforms. I find myself wondering, if there were a filter mechanism for all sites that apply practices that are in the end hostile to my brain, would I actually be missing out? And even if that were the case, wouldn't I be better off anyway?

It is also somehow true on a greater scale of things. Currently Google is attacking the open web with a proposed API that will serve to stifle browser competition and lock out users who don't like to be surveilled by ad-tech. And the best defense I can currently think of is also not technical, but regulatory politics (as this also constitutes a crass violation of GDPR). I know, they ditched "Don't be evil" as motto years ago, but it is astonishing how far they want to go to own your eyeballs in order to extract ad revenue.

So this is the reboot of the weeknotes. Maybe this time the streak will last more than six weeks. Chances aren't bad. My little daughter has adapted well to kindergarten, still for the next three month I'll work part-time only, so I may have a little more time to spare than otherwise.

Recently I dove into the topic of full-text search and information retrieval. I've toyed around with lunr. For a small codebase (around 1800 SLOC) lunr has a good deal of great ideas in it. I've written about how I gauged the code on a high-level, but if time permits, I'd also like to do some deeper dives into the datastructures and algorithms that are used.

Also I'm currently making a static site generator custom to my needs as a replacement for the messy scripts that I currently use to build this site. The sad state of that rewrite is, well, a text-book example of the Second System Effect. So, with that, I'll use the rest of my free time today to try to get it to feature parity with my messy (but working...) scripts.

What I like to do, in order to get a high-level overview and an idea on which parts I should focus, is to look at the SLOC (source lines of code) metrics. Now, this metric has a bit of a bad reputation, which stems from the days when it was used by the wrong people for the wrong reason (an in: by managers to measure developer productivity). I don't want to waste too much effort on elaborating why this is a bad idea, just so much: that is a textbook example of Goodhart's law in action.

Anyway, SLOC is easy to understand, quick to gather, agnostic of programming language, and a pretty good proxy for complexity. There is even research that suggests, that SLOC and the more complicated to measure Cyclomatic_complexity metrics, actually measure the same property.

There are several tools available to do the actual counting. I like Al Danial's cloc because it has a lot of options and output formats available. Curtis Gagliardi's loc claims to be two orders of magnitude faster than cloc, but on small to mid-sized repositories, the execution speed is not really a limiting factor, the missing JSON output option on the other hand is, .

Having said that, the output of running cloc on lunr is the following:

-------------------------------------------------------------------------
File                                  blank        comment           code
-------------------------------------------------------------------------
./token_set.js                           57            146            212
./query_parser.js                        49              0            206
./index.js                               56            239            197
./stemmer.js                             27             25            171
./builder.js                             45            153            162
./query_lexer.js                         38             18            153
./stop_word_filter.js                     4             30            131
./pipeline.js                            33            139             95
./vector.js                              26             76             94
./set.js                                 22             42             69
./match_data.js                          16             33             58
./query.js                               21            111             56
./token_set_builder.js                   15              2             52
./utils.js                               10             44             38
./tokenizer.js                           11             29             36
./field_ref.js                            7              0             21
./token.js                                4             39             15
./lunr.js                                 4             31             14
./idf.js                                  4              8              9
./query_parse_error.js                    1              0              7
./trimmer.js                              2             19              6
-------------------------------------------------------------------------
SUM:                                    452           1184           1802
-------------------------------------------------------------------------

But as a wall of text and numbers is not the most intiutive representation, I exported the cloc result to json (cloc . --by-file --json) and adapted the template for circular packing from the D3.js Graph Gallery to plot the output of cloc. Also, I did a second plot, where SLOC are adjusted by a factor determined of how heavily the code is commented (with the reasoning that a lot of comment indicate complex logic). The output looks like this:

Keep in mind, the intention of it was to be a quick gauge of a small code base. Which means it is not too economical to play with visualization features too much. One can see that the core data structures (drawn in red, especially index and token set) account for over 20% of the code base, but also the query language (in light green), which actually is a mere convenience feature and would actually be omittable, make up a similar portion. Also notably, the stemmer, which is a crucial module to actually build the index, is a inlined third-party implementation of Martin Porters stemming algorithm. In the second visualization this gets dialed down in size a lot, because the implementation does not make much use of comments, although it is very dense, full of regexes and short variable names.

But the fact that a map mustn't be confused with the territory doesn't render maps useless.

Something that I regularly think about, almost everytime I select something to read through for furthering my knowledge in my field, is if there is something like a canonical set of reading material.

On the other side of the Atlantic ocean there is a concept of a liberal arts education that is called Great books program, a four year college degree that solely consists of studying a canon of Western philosophy and literature, without a major though of its immediate utilitarian value.

As much as I find this notion of education to be romantic, I won't suggest that it would be a blue print for a technical discipline like software engineering, if only one would come up with the right set of text books and papers as canon. But I think it would be helpful, especially for those who are not at the beginning of their career anymore and outside of a formal degree program, to have some structured guidance, to maximize the value you get out of the time invested. The higher the delta to your current knowledge the better, or with the words of Alan Kay: Perspective is worth 80 IQ points.

When I searched for reading lists and comprehensive bibliographies, I stumbled over a list of Seminal Papers in Software Engineering published by members of the faculty of CMU. In the introduction they write: To understand the context of current research, it is essential to understand how current results evolved from early fundamental papers. These classic papers develop timeless ideas that transcend technology changes, and the ideas embodied in their solutions often apply to current problems. Of course, you have to keep in mind that this list was compiled in the context of post-graduate education and is a means to an end (informing the original research the students are supposed to do), which imposses a limit in size.

Imagine somebody who does not want to go down the PhD path and has already written a fair portion of software. Which techniques and ideas should such a person study? What material would sustain the remaining 20 to 35 years of their working live in software the most? I suppose the "Great Ideas in Software Engineering" would not only comprise of books and academic papers, but also certain code bases, blog posts and probably even recordings of talks. It would inherently have to find a balance between practical and idealistic aspects.

As I didn't manage to find a compilation of material that goes into that direction, I am still not sure how much merit the idea of such a curriculum does have, yet I also cannot get rid of the feeling, that many great ideas are already out there, burried under a lot of noise, waiting to be (re-)discovered.

No website is guaranteed to exist for ever, not even for a timespan that significantly outlives its original author. Keeping one up and running is, although very cheap, not entirely free. But even loss-leader offerings like github pages or the aws free tier are payed for by somebody. And that somebody might stop doing so for many reasons.

The web is a young medium, yet in the three decades an incredible amount of content has been created and discussed on it. Some of which had deep and lasting impact. Yet, there is a high risk that a lot of what was going in and on the open web is vanishing at an alarming rate without a trace from our colletive intellectual and cultural history.

Take for example Ward Cunningham, the inventor of Wiki (both the concept and its first implementation). The WikiWikiWeb/C2 Wiki was frozen in 2015. Its content is in read-only mode ever since. Since then Ward is remodelling its software basis, but in an issue on github he wrote:

Perhaps I should explain why wiki worked.

I wrote a program in a weekend and then spent two hours a day for the next five years curating the content it held. For another five years a collection of people did the same work with love for what was there. But that was the end. A third cohort of curators did not appear. Content suffered. I had visualizations. I could see it decay. That is what I mean when I say that the those of good will have passed. A security engineer has compared the open internet to the summer of love. It was neat while it was happening but it is over.

I find this stance disheartening, albeit understandable. And the c2 wiki is at least still readable. But for how long will that remain so?

There are many, too many examples of content becoming inaccessible or even destroyed on a whim. The highest risk bears everything that is hosted by commercial entities for "free": think for example of Yahoo killing Geocities. This free (as in beer) is penny-wise and pound-foolish. Single copies are single points of failure and hopeing somebody will keep the lights on is not a viable backup stragegy.

Speaking of backups: One of the very few organization that tackles the problem of preserving the web is the Internet Archive. Sadly, it recently lost a lawsuit against big publishers, which has the potential to completely destroy it financially. And it appears that there is no meta-backup strategy in place.

What can an invidual do? Those who create websites might want to consider to follow Jeff Huangs approach and design their websites to last. Probably another good idea would be to explicitely use a license for free cultural works to allow content to be mirrored without legal headaches.

As much as I'd like to end on a high note, it is obvious that such issues cannot be tackled by an invidiual or small idealistic non-profit groups and the economic incentives are not in the favor of an open web.

Since I started my website, I published 38 posts (this one not included), which means on average a little more than once a month, but my publishing cadence is far from being that regular. In some months I managed to put out something on a weekly basis, but I've also had dry spells that lasted for several months. All the while I wrote notes privately on a more or less daily basis. This makes me wonder what causes my disconnect between the frequency of writing and publishing.

At the end of the first year of this website I tried - without success - to publish daily for a month. Derek Sivers, did a similar experiment with much better success has written a retrospective on his experience. He says he the he disliked the daily publishing cadence, because it made his writing worse and with it he broke a principle, that anything he posts is really worth the time of his readers.

On the other hand, there are people who consistently publish on a daily basis over many years and sometimes even decades. Seth Godin is one of them. He writes:

For years, I’ve been explaining to people that daily blogging is an extraordinarily useful habit. Even if no one reads your blog, the act of writing it is clarifying, motivating and (eventually) fun. [..]

What I’ve found is this–after people get to posting #200 or beyond, they uniformly report that they’re glad they did it.

- The first 1,000 are the most difficult

On the question what constitues an appropriate blog post, Mike Crittenden, who also publishes on a daily basis, argues that atomic blog posts are fine. He also says: There’s space for the crappy short blog post. It’s the crappy long blog post that shouldn’t exist.

Another person who makes the case for smaller blog posts is is Matt Gemmell. He thinks that the rise of social media had detrimental effects on the psychology of bloggers:

On social, content of any length at all is fine [..]. So, perniciously, our eager-to-simplify brains have decided that the converse is true for blogs: you can write only longer, weightier stuff. The sole possible outcome of this is the one I see all the time, which is that people are reluctant to start blogs because they think they don’t have enough to say. Those who do blog will often sit on pieces for too long, because they’re waiting until they have more to say — or they shelve pieces entirely, wrongly believing they’re too brief and thus somehow trivial.

Collectively, this is all a surrendering of creative output to the whims of external platforms, and is antithetical to what makes the web so useful and potent.

- Write less

Michael Nielsen in his article on his motivations for publishing his thoughts, states that his goal is to enhance his abilities to think and create. He says that he thinks better when the stakes are higher and that one of the best ways of raising the stakes is to make a document into something you're sharing with people whose good opinion you desire.

Reflecting on my personal approach, I think my reluctance to publish more often has to do with an internal belief system that holds quality as more important than quantity. But I am coming to think that this might be a false dichotomy. To get to quality one needs to practice. Public writing seems for me to be in a different category than private notetaking. So to get to quality there is no substitute for publishing. I don't intend to force a daily cadence, but I probably will try to take Matt Gemmells advice to [w]rite less, and be at peace with it., and to do so more often.

Last year one evening my whole family was asleep unusually early. On contrast, I was not particularly tired, which is equally unusual. I used the time that I unexpectedly had at my disposal to write a tiny web app for managing my bookmarks.

Initially my goal for this article was to describe the app and its making on both a technical and non-technical level, but I got a bit lost in discussing and justifying technical minutiae, which aren't that essential. What follows is an attempt to describe the psychology of writing a simple, yet useful system in a very limited amount of time.

An itch to scratch

I regularly jump between multiple browsers on several physical and virtual machines, and I bookmark quite a lot. And if it weren't for the further, I'd be happy with the browser built-in bookmarks. I never really found a solution for keeping bookmarks in sync that suited my taste. Still, my requirements are simple. I just want a single place where I can reach all of my bookmarks, and a way to add a link with a single click.

Certainly, bookmarking-as-a-web-app has been tackled many times over. I could have taken the sane persons' approach and just subscribe to some SaaS, or I could have searched for some open source solution that to host for myself. But then again, I'm a programmer, and it seems this comes with a déformation professionelle that got the better of me. I like to call it inverse vandalism (when vandalism is destroying things just because one can, then inverse vandalism is making things because one can). Those who want to take a look at the code first can do that here.

A few foolish assumptions

Feeling the compulsion to open the IDE and start hacking is the one thing, but there is one very hard limitation: having a chunk of uninterrupted time with good attention span available. Most of that is sold to my employer already. So when I have a good timeslot, in order to get something into a working state, its complexity must be rigorously limited. To keep the program as small as possible and overcoming the the hardest limit of all, the available tiem, deciding what is out of scope is probably more important than programming the actual features. And define a certain class of problems out of existence is a remarkable productivity boost.

Many, if not all, of the assumptions I took are unthinkable in a commercial setting. Yet they turned out to be liberating in this little project. The most fundamental was, that I wanted to create a system, which is intended for me alone to use. This freed me from a whole lot of yak-shaving. Assuming a single user means I don't even need to explicitly model it out. It made it easy to estimate how much (or rather how little) memory my data will consume, and as a second order effect, suddenly the filesystem is justifiable as a good enough storage mechanism.

So what remains to have a most rudimentary bookmark manager? A mechanism to add a url, to delete it again and to render the collection of all links. All of which is accomplished by a single node.js based script which reads in a file of all bookmarks into memory and provides three endpoints. One to render all bookmarks in a single page, one to add another bookmark to the file and one to delete a single bookmark. The endpoint to add a new link is meant to be called via a bookmarklet.

Of course, a system based on those assumptions is tautologically not able to scale. But the alternatives would not have potentially scalable software vs. the scrappy single-user system. It was either something I could do in a single good session, or not creating something at all.

Lessons learned

Hillel Wayne dedicated an issue of his fantastic Computer Things newsletter to the question What can you code up in an hour?. He writes Fast programming is a distinct skill from software engineering. Software engineering principles don’t totally map over. - and this matches quite a bit with what I learned from writing this little script (and running it without major maintainance ever since).

Don't hesitate to rigorously limit scope. Make an art form out of cutting corners. Minimize the number of moving parts. YAGNI, really.

If the project is not an exercise for the sake of learning a new technology, use familiar tools. The standard library is your friend.

Most best practices are trade-offs, which, if a certain time/complexity threshold is not reached, might not pay off.

Data > API > UI. If I ever wanted to evolve what I have, the only thing I would definitively want to keep are the over 1000 bookmarks I've collected.

Learn to recognise chunks of good time as they come and grasp them firmly. You don't need anybodys permission to experience the joy that comes in creating something for yourself. Just keep in mind, when project time collides with in-person time with your most important people, that the tail end comes faster than you realize.

In light mode, I go with the user-agent styles for the links, and there the contrast ratio is fine, but the same choice in dark mode, was under the level of what the W3C Web content accesibility guidelines recommend.

My go-to resource for figuring out the right colors with regard to the WCAG is the WebAIM contrast checker.

What up until now I haven't understood was how these figures are actually calculated. And it is actually all laid out in pseudocode in the WCAG recommendation. But since it is spread out, I decided to write it up for my personal reference.

So, the spec defines the constrast ratio as

(L1 + 0.05) / (L2 + 0.05), where L1 is the relative luminance of the lighter of the colors, and L2 is the relative luminance of the darker of the colors.

To answer the latter: the lighter color is the one with the higher relative luminance. And to answer the former, I quote the definition from the WACG again: The relative luminance is the relative brightness of any point in a colorspace, normalized to 0 for darkest black and 1 for lightest white.

But how do I get to this number between 0 and 1? Well, by simply reading on in the spec. There is some some pseudo code, which is easily translatable to Javascript (which I'm doing here in form of a literate program again):

Now, the contrast ratio is easy to implement:

An input type color will return a string in the form "#RRGGBB". To use this as an input for the color parameters of the contrastRatio function I need another utility function.

So now I can drop in two color pickers: and to cobble everything together:

The contrast ratio of the chosen colors is

: 1

Coming back to the starting point, color contrast is only one aspect with regard to what affordances links should provide, but that is a topic for a different blog post.

Media queries make it very easy to detect the readers preference and CSS custom properties help keeping the rest of the styling clean and non-repetitive. So I thought it would be a right neighbourly thing to add the following:

It looks plausible to me, but visual design isn't my strongest suit. So if someone, who prefers dark mode, has thoughts on it, I'd be thrilled to hear from you.

So, did you know, that when you type the words do a barrel roll (without quotation marks) into Google's search field, the page with the results does as you bid, and turns over 360 degrees.

The implementation of this behavior can be achieved by adding a little style to the body, for example the following class:

If you want to try it out on this very website, you can use this input field.

The necessary logic to apply the styles is provided by this script:

Now, these little snippets of CSS and Javascript are not the whole point of this article. More interesting is, in my eyes at least, that what you just read is a literate program.

The code you read is not only a mere nicely formatted textual representation, but is the actual HTML style and script elements that control the rendering and behaviour of this website in your browser. This is achieved, by and large, by making the the script and style elements block-level elements, which makes their content visible. The tags themselves are rendered as content of the :before and :after pseudo selectors.

I've stolen adapted the styles from Colby Russell, who created and documented a whole static site generator in this programming style.

Another nerdy application of this idea is the naked, brutalist HTML quine by Leon Bambrick.

Update after first version: Manuel gave me some clarifying feedback via mail, and stated that he in fact the doesn't consider all internal linking as bad. My misunderstanding originates in my interpretation of his statement, that You should only use the first type when strictly necessary - which I overinterpreted as referring to something like standard navigation elements. I'll keep the rest of my reply unchanged, as I want to elaborate on the usefulness of (pontentially dense) internal linking as a design concept beyond commercial aspects.

Manuel identifies the world of marketing, search engine optimisations, and analytics as driving force of internal linking. And that might indeed be one reason. As I do not waste any time with SEO, never had analytics in place and don't plan to, I don't market anything to anyone, for I don't want to sell anything to anybody, I don't know too much about such techniques, maybe excessive internal linking in fact is a symptom of these all motives.

While I don't share these motives, I still purposefully link my former writing. I do so out of a different, and - as I hope to convince my esteemed readers - legitimate reason.

See, most of my writing starts in my private Tiddlywiki instance. Wikis as a writing medium are terrific. I've used more than half a dozen wiki softwares in my life. From the top of my head: MediaWiki, PmWiki, Zim, Githubs built-in Wiki, Redmine, and - tell me you are an enterprise application developer without telling me that you are an enterprise application developer - of course Confluence and Sharepoint. What any wiki software worth using makes trivially easy by design is internal linking. In a wiki there is on a philosophical level never such a thing as a dead internal link, only a page of which gently prompts you to write its first version.

Now, even though my website is only the subset I feel comfortable to share publically, I do like to keep the internal connections wherever it is possible. These internal connections help me to see patterns over time.

Michael Nielsen has published some notes on how to use a personal website to enhance your ability to think and create, which is kind of a manifesto for his site. Here are some excerpts of this note that resonated very strongly with me (emphasis mine):

The purpose of the site is to help me think well. This purpose is in opposition to many widely-used norms in online writing.

[..]

Depth over engagement and audience-building.

[..]

With this site I can design an interface that better supports my work. For one thing: the site navigation and organization should help me generate new ideas, and help me see the structure in my work. This site is an interface to my past thinking and possible future creative work. This means the organization and design of the site are first-class objects. As my understanding changes, so too will the organization, since the organization expresses that understanding. Thus, I should expect to reorganize and consolidate notes often. Both are a type of meaning-making.

So, yes, I admit that my internal links have a self-serving motive, as they enhance my capabilitiy and my thinking. But that is certainly not at the expense of the rest of the world. I still very much desire to help where I can to amplify the good parts of the web. For all cynical and defeatist takes aside, there is still a lot of that out there.

So that aim in mind, I've decided to treat internal and external links from now on a bit differently regarding their affordances. Internal links, that aren't part of the main navigation, will look just like the standard text and only reveal them as links when someone hovers over them, while all external links will always look like a proper link (I haven't yet figured how I will handle it on the RSS feed though, maybe in the HN endnotes style of linking?). As I consequently use relative links for everything internal, it was quite easy to achieve with a few additional styles:

article > * > a:not([href^="http"]) {
    color: #555;
    text-decoration: none;
}

article > * > a:not([href^="http"]):hover {
    text-decoration: underline;
}

And since I am on the topic of linking. In the previous article I forgot the hat tip to Curiositry, who shared a tip how to link to a segment of a youtube video both with a start and end time. I long thought that youtube had removed that feature. As turns out, only in the standard viewer, but not in embedded mode. I hope no product manager on the other side of the big pond reads this, for such a useful feature will then likely be killed before the end of the year, which would be a pity.

Given that Big Design Up Front and the Waterfall process were prevalent notions in introductory courses on software development for a long time - to some extent, they were still present in lectures I took during my studies, over two decades after this recording - that was a great question. I just love the answer that Hal Abelson gave:

I bet that's the axiom of someone who hasn't implemented very large computer systems. [...] In general, I think people who really believe that you design everything before you implement it, basically are people who haven't designed very many things.

But what may be a fallacy to most of those who actually write software - Uwe Friedrichsen elaborated on it very clearly in his article The Industrialization of IT fallacy - is still something that I see actively pursued and pushed by a certain type of executives. Of course, it's the 21st century and its a SaFE bet that we are all more or LeSS agile. But if you are looking for today's waterfall - apart from terrible puns on waterfall-like processes in the previous sentence - you often just have to take a look at the org-chart. Those who paid attention to Mel Conway's observations on system design from 1968 know, that organizational design influences the shape of the software that gets build.

I did nothing like that (yet), just another site with a little over a dozend links, in no particular order, to mostly personal blogs and websites, which I read and of which think other people might like them as well. But at least, now something exists, to which I can tend to, when the time budget permits.

This book was was edited, printed and marketed by a reputable German publishing house and bought by my mother in 2000. Neither production, nor purchase involved any kind of irony or hipster humor. And more than that tells something about some generational viewpoints and tendencies, it tells a story about a web that felt and was different than today's web: The dot-com bubble had not burst at that point in time. Neither was Wikipedia founded, nor had Google yet become an eponym for search. Search results were incomplete and lacking. Finding information was far from feeling like a solved problem.

Fast forward only a few years and everything had changed. Search quality improved to a point, that I started to form the assumption that I won't ever need bookmarks anymore, for I will be able to retrieve any information I want with a simple query. Just, that wasn't quite the end of history. Today both retrieval and content discovery seem to be problems that are look for a new solution. For the open flank, in the light of the stellar rise of capabilities and general availability of Large Language Models and the perverse incentives of SEO, is trust.

There is infinite storage for content, but the time to wade through it is quite finite. So whose recommendation do I trust? The service that delivers results that are okay-ish, just good enough capture my attention and promptly sell it to the highest bidder - or a person I can converse with, that might have a reputation to lose and who probably has already demonstrated their sense of taste and style to me in their writings?

Certainly, from the demise of Yahoo's directory we can draw a lesson that human edited directories don't scale with the amount of content available (which is of course trivially true). But why on earth would a sane person be interested in the whole web? What I want to see are the parts that still are a labor of love and conviction.

If I were to create a directory, my first editorial principle would be that the content must be be worth my time. The second principle would be, that visitors shall be treated like humans, and not as a bunch of suckers from which one might be able to extract a few cents (as beautifully illustrated here).

Coming back to this little web directory in printed form, with 23 years of hindsight, it still manages to convey two things to me: one is that preservation of digital artifacts is a hard and pressing issue (itself serves as an example in two ways, as you see: the big Yahoo directory is gone for good, this book is still there, and of course it is largely useless for as many of its URL have rotten away) and secondly, content curation is an act that creates value. Albeit, if the price tag of 29,95 DM was an appropriate expression of that value might be a point for a whole different discussion.

If you ask yourself, why you should write online, for you fear you won't be read, Manuel Monreale has an answer. He offers to be your first reader. Chris Hannah also decided that he'd like to read your blog. I share Chris' opinion, that the internet would be a more interesting place if more people blogged.

In the spirit of Derek Sivers' leadership lessons from a dancing guy: Three is a crowd and a crowd is news. - If you start a new blog, or already have one and your only visitors are bots: send it to me, I'd love to read it. If you provide a feed for it, even better, I'll add you to my feedreader. And when I finally find some time to create a blogroll, you'll be in there as well.

Indeed, Bill has written a lot in nearly three decades. In the answer to the question how he managed to create so much stuff, one phrase struck my particularly:

I don't live in XP or ME or Windows 95. Not win3.1 either. Not MSDOS. I live on the internet, in an ancient unix shell account which is aliased to look a bit like msdos. If I should ever type up a little textfile during other activities on the computer, it only takes me ten seconds to put it on my webpage. Unlike most people, for me the barriers against publishing on internet have entirely evaporated

Isn't that a beautiful idea? The web as a basically immediate writing medium, with basically nothing stands technically in between writing something down and having it published. Alas, rather few people live out their intellectual life entirely in a unix shell.

For most the barriers are firmly there, the friction is as real as the perceived need for access to the web as a medium of personal expression.

Many of today's walled gardens got big, because a part of their initial value proposition was to lower the barriers of entry for many. They generally didn't do that with the public good in mind, and the quality of all those platforms deteriorates in a cycle, that Cory Doctorow has recently sketched out, and coined the colorful and sadly adequate term enshittification for.

I haven't figured out a solution for myself that I would perceive as frictionless yet. This website started out as a single static html file, something that would have indeed been manageable via ssh in a shell. But frankly, while I find my way around vim, I don't particularly enjoy the experience. Apparently, I am not enlightened enough and I like proper UIs maybe a bit too much. So what I have is a Tiddlywiki in which I write down most of my notes. And while I love Tiddlywiki as a writing medium, I don't want to expose all of my notes and I don't want to serve a metric ton of Javascript for completely static content. So now I have, apart from my wiki instance, a few horrible scripts that sit atop of my private wiki instance, filters the parts that are meant for publication (as indicated by tags, which are a very useful first-class primitive of tiddlywiki) and transforms them to some static html. But still, a wiki remains a comparatively non-mainstream interface for writing and the site generation script is a one-off, and I still haven't completely automated the feed generation with that approach. The whole problem it certainly quite a yak to shave, but probably - hopefully - worthwhile.

The web, with its not quite 2⁵ years of existence, is still a young medium. Blogging as one of its archetypical forms hasn't actually been away. Yet, that the web is an open platform is so not through a law of nature. Given the economical and political circumstances of surveillance capitalism, and the maddening drift to authoritarian forms of governance, those properties would better not be taken for granted.

But that is all the more a reason to experiment, tinker and shape the medium through usage and many more cute little experiments, and do what is possible to increase the signal-noise ratio of the better corners of the web.

All that is needed to make it work is a single link tag added to the head. It has three mandatory attributes: href of course for the URL of the feed, rel with value "alternate" and type with a value of either "application/rss+xml", or application/atom+xml, depending on the feed format. The title attribute is optional, but it's useful if a site offers multiple feeds.

In quite a lot of cases it works like a charm. It used to work here as well. Now, guess which self-proclaimed fan of feeds thought it would be a good idea to rewrite the template of his personal website and managed to forget putting in the following line into the head:

<link href="proxy.php?url=https%3A%2F%2Fholzer.online%2Ffeed.xml" rel="alternate" type="application/rss+xml">

The approach has one problem: it is difficult to extend the API without introducing a breaking change. Two common features, which can typically be omitted in the first version, but are hard to retrofit when the returned type is an array, are metadata and pagination.

The simple prevention is to wrap all collections into an object from the beginning. Existing clients can then simply ignore all new properties, if they need to be introduced, whereas a change of the type of the response from array to object is guaranteed to break those clients.

I love personal websites (as opposed to the corporate web) and the idea of Bring Back Blogging resonated strongly with me. I just signed up to it. The project: publishing three articles in January (+an announcement at the beginning and the end) and providing a RSS feed for the site.

So this is the announcement of the beginning. I'll keep it short, and will use the rest of the evening to get my RSS feed generation script in order again.

So I prompted it to write a bash script which plays tic-tac-toe using an optimal strategy.

In the first response, I already got quite a lot of building blocks:

• a data structure for the board (more about that later...)
• a function to print the current state of the board
• a function to check for a draw
• a function to check whether any player has won
• a function to check for whether a move is valid
• and the beginning of a function to decide the move of the program.

I assume the API ran in some sort of timeout, as the last function wasn't implemented fully. Therefore I prompted it a for the whole function to make the computer move, and it indeed delivered complete implementation. So I had a lot of functions, but no game to actually play. And naturally - or at least naturally for software developer - I asked if it can generate a game loop, and I wasn't let down.

I copy-pasted the three snippets into a file, held my breath, executed the script, and: it worked. Well, at least sort of. The generated program let the human player make the first move. As the only rational opening move is to take the center field, I didn't immediately discover a subtle bug, until I tried to play a non-optimal game.

The generated code represents the board as an array of nine strings, where an empty field is represented as a single space character. The array was only initialized with 8 strings. To be fair, there are only two hard problems in software engineering: cache invalidation, naming things and off-by-one errors. And who am I, to be too harsh in a code review? I conversationally (it's ChatGPT after all!) asked it whether it could spot and correct the off-by-one mistake in the initialization of the board. And here it started to become obvious that it is a gigantic correlation machine, which although it produces very impressive results (and confidently sounding one's), it has no actual concepts of the things it generates. There is - as expected - no intelligence and no learning in the human sense. Here is how the dialog went down.

You are correct, there is an off-by-one error in the initialization of the board. Currently, the board is initialized as follows:

board=(" " " " " " " " " " " " " " " ")

Just that the board wasn't initialized as quoted above. This was no reproduction for its initial error, but another one (which it continued to explain in prose - and incorrectly so).

This creates an array with 13 elements, with indices ranging from 0 to 12.

After that it created a description of a correct solution in prose, directly followed by yet another incorrect implementation:

However, in the tic tac toe game, the board has only 9 squares, with indices ranging from 0 to 8. To fix this off-by-one error, the board should be initialized as follows:

board=(" " " " " " " " " " " ")

This creates an array with 9 elements, with indices ranging from 0 to 8, which corresponds to the 9 squares in the tic tac toe game.

It is a fun tool to play around with and I'm not unimpressed by the results. But I don't tremble in fear of being layed off anytime soon due to its existence. For those who are not convinced, Hendrik Erz has written up a lot more good reasons not to fall for the hype.

If you are interested, you can also read the complete chat transcript and the generated source code (where the correction of the initialization was done manually).

Lately I was thinking about the relationship between learning, teaching, schooling and curiosity, especially with regard to teaching computing. I cannot pin it down to something concret, but I strongly sense that somewhere in the system mechanisms are in place, that are detrimental to curiosity and form learners into consumerists.

Therefore, one thing I'm looking forward to is the upcoming round of DevEdBookClub. An informal online discussion round, that will discuss Greg Wilsons Teaching Tech Together over the next few month. I've read to book earlier this year, and it is well worth to be re-read. As I did not take notes on the first reading, this is a great opportunity not only to discuss it with other people, but maybe also get a summary done.

I've also been thinking a bit about my writing process. The fixed cadance of weekly notes has a liberating aspect: when something doesn't make it in there, there is a place for it next week, or it might even not be important after all. But having a topic for an article in mind makes it easy to move the release further and further, in the worst case until I don't see a point in putting something out anymore. I'm asking myself, if artificial deadlines might be not evil after all? Maybe I should budget a certain amount of time for a given topic I'd like to write about upfront and then just put out what ever I have once that time budget is used up?

And since the timebudget for this week's notes has basically run out and it's quite humid and hot in the appartment, I'll seek a place with more shadow now...

I've also thought a bit about how a both practical and principle-oriented introduction to software development would have to look like. Could I outline what I think to be essential? I tried and ended up with a whole lot of disparate, little aspects - it is really alarmingly easy to miss the forest for the trees. I sense that there is a lot of potential for simplification overall, but as the saying goes: simplicity is the ultimate sophistication...

Another ongoing topic of mine is coming to terms with a lot of parallel ideas and loose ends. I've started thinking in the direction of serving on-going writing in a separate folder as raw plain text. That would be quite frictionless, but it I have to make up my mind about a few details. Would I want to keep URLs of such draft/work-in-progress files stable, shall they be first-class citizens of the site, should they be indexed, how would I indicate freshness/state etc.? Also I started to think about a general restructuring of the site. After all, I've strayed away from my initial principle, that every piece of content shall live in the index.html.

And - third time's the charm - this weeknote format seems to be working for me and my schedule.

An attempt of simplification, which results in bloat, is a perverse outcome from an engineering perspective. The engineering lense is of course not the only one, but a system that is to be maintained in the long run, surely shouldn't completely disregard it. For what could be a second-order effect of such a decision?

In the worst case: the creation of an incentive be silent. A motivation to stop contributing suggestions, and sweep issues under the mat. Such a potential breakdown of communication and feedback is a real threat for the quality and structure of any system. If a canary in the coalmine stops singing, miners know that it is time to get out.

Those who are tasked with creating systems would do good to be reflective of this, and if necessary take measures to counteract it.

One of the perks of having a personal website is that I can play with web technology as much as I want. For example: I'd like to add support for webmentions, but without a third-party service. While the specification seems straight forward, I'm a programming long enough to expect the devil being in the details of even the most straight forward specifications (And life is short, while art is long.).

I also thought a bit on recurring design elements of personal websites in general, which led me down quite a rabbithole. I've collected nearly thirty of them by now and I don't think that list is exhaustive yet. This is a topic that would deserve a proper write-up. Among those elements is for example the blogroll. I think that would be something I should add one to my site as well. I believe we'd better spend more time on personal sites instead of lingering in the gated communities of the web.

On Work-In-Progress

John Cutler shared a checklist on Twitter that resonated a lot with me. It is not only applicable in a work context but in the private life as well. Even if I think about this little corner of the web, I do have a few things that I've started but not yet quite finished...

Is your organization struggling with too much work in progress? How would you know?

- John Cutler (@johncutlefish) May 22, 2022

But more pressing than gauging the question of if, the is the question what to do about it? Sadly, just creating a Kanban board in JIRA does not quite cut it...

What aspects does a good life have?

Also via twitter I came across the article A Psychologically Rich Life: Beyond Happiness and Meaning by Eron Westgate and Shigehiro Oishi. I think their finding, that happyness, meaningfulness and "psychological richness" of life are and can be measured as separate categories, as an enligthening mental framework. It resolved for example a cognitive dissonance I had, when I first read a certain passage of Richard Hammings talk You and your research, where he said that he had neglected his spouse in order to to get what he wanted done. Happyness as category can be in tension to meaningfulness or psychological richness. All aspects contribute to a good life, but each is doesn't weight equally for everyone.

Snowclones in technical articles

Sometimes titles sound strangely familiar. They follow a certain phrasal template which sounds a bit clichée. Such templates also go under the name snowclone. I think I've encountered at least half a dozen of them in the context of computer science / software engineering. It might be worth an article to collect them.

That's not quite all folks...

There are certainly more things to be said, but as I want to put something out at a predefined time, as artificial as that might be - so I'll close it here, and in the next notes of the week I may muse on some of the following questions:

• What can be done to make software development less of a red queen's race?
• Might it be worth to take up long letter writing?
• If both time and money are constrainted, what is the most effective way to learn a challenging subject/topic?

A colleague of mine recently reviewed a part of a UI component library I worked on with regard to accessibility. He found a line where I used a <b> tag instead of a <strong> tag and rightfully remarked that I'd better use sematic tags.

After the review, I thought about the reason I came up with the non-semantic markup in the first place. The component was developed against a designers' mockup, and when I derived the markup I translated the visual design in my head to the idea, that this part should be formatted bold, from that my mnemonic association I went straight to the <b> tag.

Satisfying a specification from a mere typographical point of view is not enough. What makes no difference for visually unimpaired people very much matters for users of assistive technologies.

Sure, I‘d be hard pressed to come up with a category much more generic than „Making things“. Yet it rules out mainly consuming activities (e.g. reading books, visiting an exhibition, traveling, etc.). Those should go on different lists, but what to actively include definitively needs some thought. When that thought enables you to focus once you do have time to allocate, it would be well-spent. Dabbling around without a concrete goal likely results in spreading yourself too thin and the top-most item of a maker‘s bucket list should act as a deterrent to that kind of behavioral pattern.

A few questions to ponder, when creating your list can be:

• What is outside of your immediate needs that you would find fun?
• Which branch your field would you like to explore?
• What would you wish to have come up on your own with?
• What object would be cool to have?
• What was the last thing a single person came up with that I admired?
• Which off-the-shelf product does an okay, but not very good job?

Personally, I‘d also love to read such lists on the websites of creative people. I have to admit, my own list is not in the best shape. Feel free to call me out for being a hypocrite if I don‘t put mine on here some time in the near future. But since I see my current top-most item staring at me, and I‘ve got a few minutes left, so I know what I‘ll do after this little piece is published.

The polar opposite of these walled gardens are the many websites (mostly still operated by individuals) that still offer to subscribe to their content via a feed. Feeds empower the readers by decoupling the content consumption from the platform. No algorithm orders it for you, just a simple timeline. And a much slower one for that.

I have roughly the same number of feeds in my feed reader as I follow people on Twitter. Yet, I refresh the former manually once a day and have maybe a dozen articles in the inbox, spending a mere five minutes on the latter I'm being nudges to refresh and see more than fifty new items. Sure, your mileage may vary and the the plural of anecdote isn't data - but that's roughly a three orders of magnitude difference in items that are presented.

With regard to that, I overhauled my own RSS feed. While the first solution was a fine exercise in Satisficing, it had limits. It only delivered the last five articles to new subscribers and didn't serve the full content. The new one does all that and doesn't depend on a third party to create it.

I don't know how many years I typed "cd" in the terminal to change my directory, until I learned that the shell can interpret anything you type, which is no command, to execute implicitly as argument to cd. I enable this by setting an option, with adding the following line to my .bashrc:

I hope it might save a few keystrokes for whoever reads this.

Having a regular markup structure made it possible to an external service derive the feed automatically. So, here you go.

To avoid falling prey to the collector's fallacy, you need to work actively with the collection you have. Introduce some form of rate limiting: Don't buy a new book before you finished one. Don't bookmark another article before you haven't read (or at least deleted) one of those you have bookmarked.

If it would have been important, you would have already read it. Its content likely won't be lost. There are search engines and even in case the of uncool urls there's still the wayback machine.

Now, there is no copyright to that term, but one overarching concept I see and very much like is, that articles are not seen as something that is written once and then finished after the publication, but as a seed which might grow into something that eventually becomes evergreen, more akin to a well-organized public (read-only) wiki.

I haven't yet figured out how I will this page in the long run, currently it is much more a blog than a garden, yet I find the idea to go back to existing article and enhance them over time very intriguing.

Herbert Simon coined the word satisficing for a strategy that concentrates on getting an option that is good enough, but possibly not the very best. Satisficing is one of the foundations of productive human behaviour because it avoids wasting time on finding improvements that are not going to make a significant difference.

Yet, satisficing should not be applied to very high-priority endeavours where much is at stake. There pursuing excellence might actually be the better strategy. But those are few and far between. To have the capacity to make them happen, satisficing is an imperative.

Today's default choices are quite distinguishably different from that. A minified version of Bootstrap 4 ships 117kB. As a comparison: I would have to write about 60 to 80 additional articles on this site to match that weight with an equal portion of content. And while at it, I'd have to clutter my markup with classes over classes. There are of course slimmer css frameworks than Bootstrap, but nonetheless, my main motivation is in the writing down useful things, therefore I decided to approach the styling of this site quite radically with classless CSS. All styling decisions are taken by CSS which is targeted by element selectors only. If I'd bother to minify it, that would clock in little over half a kilobyte, and even unminified it is less than 1% of bootstrap.

But page weight isn't the most important thing. A classless CSS approach enables me to author content without intermediate representations (e.g. the Markdown flavour du jour in combination with a static site generator) directly in HTML while still allowing me to focus on the content.

Therefore I decided to add a reasonable explicit expiration date to any such TODO (some 4 to 8 weeks in the future). When I encounter a comment that has expired, deleting it is a perfectly viable cleanup. The code structure obviously was not so bad after all, or else I would have put energy into the refactoring already.

I use the following criteria to assess if a topic is worth spending my time on it.

Longevity and Transferability

I prefer evergreen knowledge and focus on topics that are not bound to a specific technology. As a general principle, anything with a version number attached to it should be treated with caution. But: Using a specific technology (even if it's non-mainstream or dated) to illustrate a concept is fine.

Applicability and personal usefulness

Many concepts are long-lived and transfer well, but when the chance of applying them is rather slim, then other things should take priority. Not applying something is detrimental to retention and time is too valuable to spend it on learning something that falls into oblivion.

As a caveat: To some degree, it cannot be known which topics will lend themselves to be applied to problems presented to you due to external needs and interests. Chance favours the prepared mind.

Immediate necessity

Not everything which needs to be learned will be a beautiful timeless idea. Sometimes I need to grasp a poorly documented functionality of some obscure technology to get a task done. It comes down to a judgement call, which is a matter of experience and confidence, how deep to go. It can be beneficial to build a real understanding instead of just copy-pasting from the web (which might get the job done but comes at some loss of control).

Personal interest and commitment

There are also a few topics and technologies, to which I'm committed deeply. I consider it worthwhile to keep up with them, even if the criteria laid out before are to some extent not met. I also regularly reassess my commitment to the topics.

If the order of development is optimizing for throughput, the so-called quick wins and quick fixes will be prioritized and the effect results in a well-known problem in scheduling algorithms: starvation.

Resource starvation is a problem encountered in concurrent computing where a process is perpetually denied necessary resources to process its work.

In this case, the process is the task in the backlog, which would be the fundamental solution to the problem, and the resource is the allocation of time to work on that task.

Steve McConnell formulated yet another risk of those quick wins and fixes in his "Software Project Survival Guide" (p. 127):

The problem with quick and dirty, as some people have said, is that dirty remains long after quick has been forgotten.

If one separates the responsibility for functional specification from responsibility for for building a fast, cheap product, what discipline bounds the architect's inventive enthusiasm?

The fundamental answer is thoroughgoing, careful, and communication between architect and builder.

The architect has two possible answers when confronted with an estimate that is too high: cut the design or challenge the estimate by suggesting cheaper implementations.

For it to be successful, the architect must

• remember that the builder has the inventive and creative responsibility for the implementation [..]
• [..] be prepared to suggest a way of implementing anything he specifies [..]
• deal quietly and privately in such suggestions
• be ready to forgo credit for suggested improvements

The builder will counter by suggesting changes to the architecture. Often [..] some minor feature may have [..] large costs when the implementation is worked out.

Brooks considers that, the first system which an architect builds is potentially clean. As the system is designed, there are ideas which get stored away to be used "next time". All the frills and embellishments which were kept out of the first design, often make it into the next system. Therefore, he states:

The general tendency is to over-design the second system [..]

and

The second system ist the most dangerous system a man ever designs.

Basic system assumptions can change. These changes make some techniques obsolete. The tendency to refine those can be another symptom of the second-system effect.

An architect cannot skip designing a second system, but he can be conscious of it hazards. Self-discipline must be exercised to avoid functional ornamentation ("gold plating") and sinking time in obsolete areas and functions.

He describes strong tensions between Robert Metcalfe and Chuck Thacker, who are both pioneers of the Ethernet and personal computing. Hiltzik quotes Bob Taylor, the manager of PARCs computer science lab, who introduces what he calls a "Class One disagreement" as the root cause.

According to Taylor, two people have a Class One disagreement when they disagree and neither can explain to the other's satisfaction the other's point of view. When each person can explain to the other's satisfaction the other's point of view, it's a Class Two disagreement. The latter enables people to work together, even if they disagree, while the former is destructive.

With that framework in mind, when you "agree to disagree", it would be wise to ensure that it is on a Class Two disagreement.

Susan Weinschenk has compiled a lot of research into a hundred short essays. The book covers a lot of ground with topics ranging from perception, over how human memory and thinking and decision making processes work to how humans tend to deal with errors.

The book doesn't require any prior knowledge. It is very digestible and actionable. The chapters are to the point and contain a condensed list of key takeaways. No reading order is required. Being very broad in topics makes it rather hard to summarize - yet cheapskates and people in a hurry could probably even get some insights out of simply reading once through the table of contents.

All in all: a recommended reading.

Meanwhile, I made a profession out of what was once my hobby. Earning one's bread doing what one would have gladly pursued free, for passion is a privilege granted to only a fraction of the human race, as Fred Brooks rightfully stated in the epilogue of his seminal Mythical Man-Month. So, by and large, I consider myself a very lucky man.

Yet, over the years I lost track of the passion, the fun aspect slipped away a bit. This, therefore, is an attempt at rekindling it. For now, in the shape of a single file, filled with random musings of some weird dude who writes raw HTML.

Just for the fun of it. :)