AnnoyScript

Adding view count and like button to 11ty

2024-06-21T00:00:00Z

I like 11ty for its simplicity. Once you understand that its documentation is shit^[1] and you are left alone in this developer life to struggle in the ultimate quest to achieve enlightenment, you might even say that it's a good blogging framework that can teach you important lessons about craving and duḥkha.

A few months ago, I was eager to add some small features to my blog: a view count and a like button that viewers can click to give me some temporary pleasure before I slide back into fullness of emptiness.

So I started my quest with simple idea: add express as the backend, create some APIs, write data to a JSON file, and voila!^🎯

flowchart LR
    subgraph static
        A([11ty]) --- B[Post1]
        A --- C[Post2]

    end

    subgraph dynamic
        B --- D[Views.njs] & E[Likes.njs]
        D & E <-.->F([express])
        F <-.-> G[(Stats.json)]
    end

The issue with this approach is that I don't have my own server, just a free Vercel hobby account. Vercel doesn't allow writing to a file and directs you towards a more serverless approach, encouraging you to learn new things instead of staying comfortable with your 10-year-old mindset.

Cooking the solution

Mixing 11ty with Express and Vercel serverless functions is like mixing cough syrup with iodine and lye^[2]. It takes time, but you might end up with something nice. Or, you could get frustrated and burn your house down. It took me about two working days to get it sorted properly, but the house is okay.

The issue lay in the Vercel itself - by default the framework is predicted from build script from the package.json file. In our case is clearly 11ty.

{
  "name": "dynamic-11th",
  "version": "1.0.0",
  "scripts": {
    "build": "eleventy",
    "watch": "eleventy --watch",
    "start": "node server.js" // this won't be executed in serverless context
  },
  "dependencies": {
		"@11ty/eleventy": "^2.0.1",
		"express": "^4.17.1",
		"@vercel/kv": "0.2.1"
	}
}

But we can override that by setting different options in the Project Build Settings. The only issue is that we cannot set there two separate build settings (static 11ty build and node execution). However, we can utilize the vercel.json file to achieve this

{
  "version": 2,
  "builds": [
    {
      // build static sites using npm build and set output directory to /_site
      "src": "package.json",
      "use": "@vercel/static-build",
      "config": {
        "distDir": "_site"
      }
    },
    {
      // start the node server as a serverless function
      "src": "server.js",
      "use": "@vercel/node"
    }
  ],
  "routes": [
    // handle all API requests to express
    { "src": "/api/.*", "dest": "/server.js" },
    // handle everything else to index.html
    { "src": "/(.*)", "dest": "/_site/$1" }
  ]
}

So all we have to do now is create and populate server.js in our root folder with some basic API endpoints and a place to store the data. Fortunately, Vercel provides serverless Redis storage: Vercel KV^[3], which we can easily utilize for that purpose.

Let's update the chart with more data so we know where we are right now

flowchart LR
    subgraph 11ty [11ty]
    C(postTitle) ---  D[Views.njs] & E[Likes.njs]
    end
    subgraph node [node]
    F([express])
    end
    subgraph Vercel [Vercel KV]
    subgraph Key [Key]
    G(postTitle)
    end
    subgraph Value [Value]
    H[likes] & I[views]
    end
    end
    Key --- Value
    F <-.-> Key

    D <-. API/JSON .-> F
    E <-. API/JSON .-> F

So now we just need to access the database and get or set the key. Let's assume that we already have postTitle, so our pseudocode can be simplified. To increase the value we use HINCRBY; to retrieve it, we use HGET.

const { kv } = require("@vercel/kv");
// Increase view
// If key does not exist, a new key holding a hash is created
// If value does not exist the value is set to 0 before the operation is performed
const views = await kv.hincrby(postTitle, "views", 1);

// Get likes 
const likes = (await kv.hget(postTitle, "likes")) || 0;

Important

Note that even though the official Redis documentation states that these functions can be written in uppercase, the code within server.js is case-sensitive!

Also, observe the brackets around @vercel/kv import. I wasted 3 hours looking for the reason why KV is not working properly.

Now the server.js can be populated with proper APIs that will handle get/post requests.

const express = require("express");
const { kv } = require("@vercel/kv");

const app = express();
const PORT = process.env.PORT || 8080;

// Middleware to parse JSON bodies
app.use(express.json());

// Basic health check
app.get("/api/", (req, res) => res.json({ message: "Hello from Express!" }));

// Get the likes count
app.get("/api/:postTitle/likes", async (req, res) => {
    try {
        const { postTitle } = req.params;
        const likes = (await kv.hget(postTitle, "likes")) || 0;

        res.json({ likes: likes });
    } catch (err) {
        console.error(err);
        res.status(500).send("Error getting likes count!");
    }
});

// Update the likes count
app.post("/api/:postTitle/likes", async (req, res) => {
    try {
        const { postTitle } = req.params;
        const likes = await kv.hincrby(postTitle, "likes", 1);

        res.json({ likes: likes });
    } catch (err) {
        console.error(err);
        res.status(500).send("Error updating likes count!");
    }
});

// Get the views and increment them. Every request is a new view
app.get("/api/:postTitle/views", async (req, res) => {
    try {
        const { postTitle } = req.params;
        const views = await kv.hincrby(postTitle, "views", 1);

        res.json({ views: views });
    } catch (err) {
        console.error(err);
        res.status(500).send("Error getting post views!");
    }
});

app.listen(PORT, () => console.log(`Server ready on port ${PORT}.`));

module.exports = app;

Now the best part. Putting it all together in the half-ass manner^[4].

In the first line of Views.njk the data-postTitle should contain {{ title }} value, but unfortunately 11ty tries to render the post title there, so I had to improvise for clarity. In your solution, simply replace it with curly brackets.

<p id="views" class="views" data-postTitle="<< title >>"></p>

<script>
  const viewsP = document.getElementById("views");
  const postTitle = viewsP.dataset.postTitle;

  document.addEventListener("DOMContentLoaded", async () => {
    const response = await fetch(`/api/${postTitle}/views`,{ method: "GET" });
    const data = await response.json();

    viewsP.textContent = data.views;
  });
</script>

and LikeButton.njk with the same issue like before:

<button id="likeButton" class="likes" data-button-id="<< title >>">
  <span id="likesCount"></span>
</button>

<script>
  const likeButton = document.getElementById("likeButton");
  const likesCount = document.getElementById("likesCount");
  const postTitle = likeButton.dataset.buttonId;

  document.addEventListener("DOMContentLoaded", async () => {
    const response = await fetch(`/api/${postTitle}/likes`, { method: "GET" });
    const data = await response.json();
    likesCount.textContent = data.likes;

    const handleClick = async () => {
      const response = await fetch(`/api/${postTitle}/likes`, { method: "POST" });
      const data = await response.json();
      likesCount.textContent = data.likes;

      // Remove the event listener after the first click
      likeButton.removeEventListener("click", handleClick);
    };

    likeButton.addEventListener("click", handleClick);
  });
</script>

This code can be accessed like so {% include "likeButton.njk" %} on every page. You can check working demo and the code on GitHub. Now smash that like button below!

I love creating and fixing documentation myself, and I'm really proud to call myself a documentation hero. However, 11ty's documentation is beyond anyone's ability to improve; it's simply bad. The only way I can see it being improved is by starting from scratch, and I just wanted to create a quick blog, not save the world. For anyone else considering 11ty, I suggest Next.js. Implementing features like a view counter or a like button is a breeze there, and you can even find ready-made projects with these features included. ↩︎
...then mixing it with strike pads off a matchbook, muriatric acid and hydrogen peroxide, and then, maybe somehow... crystal meth 🙅 ↩︎
KV stands for key and value so you can interact with this pseudo-database like a normal object, with limitations due to its very daft syntax. This was further restricted by Vercel itself to only few keywords. For those who dislike this solution, there is also Vercel Postgres ↩︎
I'm sure there are better ways to handle this in njk files without resorting to ugly <script> tags, but I've had enough of that for now. ↩︎

Web workers with React and TypeScript

2024-05-22T00:00:00Z

Annoy level	Time wasted	Solvable
	16h	Hell yeah!

One thing that particularly bothers me when working with libraries/frameworks like React is their hidden overheads. Everything generally functions smoothly, but when you delve into less common use cases, you're likely to encounter some nasty bugs^🐛.

While the reasons behind these bugs are often clear, finding a solution within the framework's constraints can be quite challenging. That's what happend to me while I was working again with web workers.

The good 🦸‍♂️

While working on my Shan Shui project, I set out to create millions of SVG paths and swiftly integrate them into the DOM. Each frame I rendered included numerous layers, ranging from 4 to around 100. Many of these layers contained thousands of SVG elements, sometimes even more. Rendering all layers simultaneously would boost rendering process significantly compared to the standard synchronous JavaScript approach.

When thinking about speed and parallel computation in JavaScript, there is only one star: web workers^⭐. Their ability to work independently from the main thread and their true multi-threading capabilities caught my attention immediately.

My first draft code simply followed the MDN tutorial.

this.someArray.forEach((element) => { 
    const worker = new Worker("worker.ts"); // worker.ts is within same folder, da!
    // ...

However, this does not work in a React project because the bundler does not understand that this specific string should be replaced during compilation with the new path to the bundled JavaScript file.

After some digging, I found a clever solution that works with React's bundler.^[1]

const worker = new Worker(new URL("worker.ts", import.meta.url));

Now, with the help of promises, I can render the entire scene just as I showcased in my GitHub Gist.

The bad 🦹‍♂️

Let's split the previous one-liner so I can explain what's happening here.

const url = "worker.ts"
const base = import.meta.url;
const path = new URL(url,base);
const worker = new Worker(path); // this is just the same code, yeah? Well...not in React

base represents the URL from which the script was obtained. More on import.meta.url.
path creates a new URL object after resolving relative references between url and base.
worker creates a new Worker object using a string from path's URL object.

Everything seems straightforward, but there's a _💥ZONK^💥 - it's not working!

The code fails with error: Refused to execute script from '<URL>' because its MIME type ('video/mp2t') is not executable. So now, my worker's TypeScript code appears to be treated as a video.

Important

According to webpack's doc: "(...) while the Worker API suggests that the Worker constructor would accept a string representing the URL of the script, in webpack 5 you can only use a URL instead. Using a variable in the Worker constructor is not supported by webpack".

This limitation exists because webpack needs to analyze the URL statically during the build process. This requirement ensures compatibility with native ECMAScript modules and scenarios where a bundler is not used.

We could just skip this way of passing the URL object and do it directly. However, if you ever consider (just like I did) creating a custom worker class, you might have a bad day ahead of you.

class CustomWorker extends Worker {
    constructor(scriptURL: string | URL, options?: WorkerOptions) {
        super(scriptURL, options); // React intensifies breathing
    }
}

The new CustomWorker(someURL) constructor will fail with the same issue as before.

The simplest solution is to use a one-liner for your project or a workaround where the worker's code is written in JavaScript (which does not cause a MIME type issue). If you really need to pass the URL as a variable, or build worker with TypeScript, you can use a trick that I learned while trying to fix another worker issue.

The ugly 🥸

While happy about my nitro-speed worker approach, I published the project on Vercel. Then, the nightmare of every engineer occurred — it works locally (almost) perfectly, but stutters in production. A quick comparison between the two versions shows a major flaw: the workers are now working sequentially! Somehow, the parallelism disappears^[2]. What took 243ms on localhost now takes 1.32s in production^[3].

It was not long after I thought, "maybe the network requests are the issue here", and I was right. Unfortunately, network requests made by Workers are not cost-free.

After a few more readings, I found this solution which relies on the Blob constructor. It consumes the stringified version of the main worker's function and then passes the Blob object into URL's createObjectURL static method.

// worker.ts file //
const workerFunction = function () {
    onmessage = function (e: MessageEvent): void {
        const element = e.data.element;
        const result = element + "!" // Just as an example

        postMessage({ result: result });
    };
}
// Stringify the whole function, getting everything between first {} brackets,
// creating a blog containing onmessage function and URL from that Blob object
const code = workerFunction.toString() 
const funBody = code.substring(code.indexOf("{") + 1,code.lastIndexOf("}")); 
const blob = new Blob([funBody], { type: "text/javascript" }); 
const workerBlobURL = URL.createObjectURL(blob); 

export default workerBlobURL;

// main file //
import workerBlobURL from "worker.ts";

this.someArray.forEach((element) => { 
    const worker = new Worker(workerBlobURL);
    // ...

I gave it a try, and it works astonishingly well. All the network overheads have vanished, and now the rendering of all the layers takes just 163ms!

Tip

When creating a worker without worrying about network request overhead, simply use a Blob and create it on the fly. The same applies to custom workers created in React - pass the Blob-generated URL object as the argument instead of a regular path.

The only downside of using Blobs is that complex objects with import statements will recreate the issue with multiple network requests. What's worse is that they will all link to a copy of the imported scripts. Therefore, when there is a need for information flow between the created elements, this solution will fail badly, creating more issues than it solves.

Further optimalization

Workers work best when you use them in line with the number of logical processors available to run threads on the user's computer. This number can be obtained by accessing the Navigator's hardwareConcurrency property. This way, we can partition the data (for example, my array of layers) into chunks of length equal to the logical processors.

function chunkLayers(layers: Layer[], size?: number): Array<Array<Layer>> {
    const chunkSize = size || navigator.hardwareConcurrency;
    const result = [];

    for (let i = 0; i < layers.length; i += chunkSize) {
        result.push(layers.slice(i, i + chunkSize));
    }

    return result;
}

...and then create pool of workers that will process each piece in the fastest possible way.

const chunks = chunkLayers(this.layers);

const framePromises = chunks.map((layers) => {
    const chunkPromises = layers.map((layer) => {
        return new Promise<string>((resolve, reject) => {
            const worker = new Worker(workerBlobURL);
            // skipped worker's onmessage, onerror, postMessage
        });
    });
    return Promise.all(chunkPromises);
});
return await Promise.all(framePromises);

In my code, this approach doesn't bring any speedup as the data is not that complex, and I need to wait for all the promises to be resolved. In the tested example, only around 50k elements were rendered, which brings the total time per element to around 4μs. Further improvement could be limited by the overhead associated with using Promises. However, for more complex processes where data chunks are not related to each other, it might be the go-to option.

Making it work in TypeScript and the WebPack bundler (without React) is another level of flexing ↩︎
Only when publishing this post did I notice that the local version was suffering from the same issue, but to a lesser extent, as there is much less network overhead on localhost ↩︎
I tested all the performance benchmarks on the same picture ↩︎

The fastest way to work with arrays

2024-03-20T00:00:00Z

Although my grandma liked to spend time feeding me sweets and telling lies, such as the claim that all methods in V8 are equal in performance, I knew that she was secretly hiding the truth of real JavaScript ninja techniques.^🥷🏻.

Even though the cake was delicious, it's important to know that a true Saiyan always sprinkles when he tinkles^[1], and the real ninja always assumes that JavaScript's mysterious engines were written by triple agents who made their stuff so confusing just as a test of our intellectual superiority and endurance before we can truly enter the Ninja realm.

Seting up the task

Imagine the situation where we want to have a workout array that contains all of our daily push-ups. And we do a lot of them. Like milions of them, and every push-up it's actually a pretty heavy-lifting for a memory.

The go-to approach by laydev would be something like this:

const workout = [];

for (let i; i < 10000000; i++) {
  workout.push("push-up");
}

Looks okay, but it's actually quite slow. And don't even try to tell me that "everything will be fixed by a V8 engine that will magically turn this lazy-ass code into performance beast". We are not in the GCC town; it's the JavaScript ghetto! Things just don't get smoothed out here so easily.

Let's find a better, totally ninja-style, blazingly fast secret technique. But first, we need to reveal the first truth about JavaScript's arrays.

Empty array vs prepared array

Should we start our workout with an empty mind or filled with memories of the times we failed as a developer? Or, to stay on track with the main quest, should we just create an empty workout array, or is it better to have it filled with empty slots ready to get sweaty💦?

const workout_empty = [];
const workout_slot_ready = ["slot",...,"slot"] // 10000000 slots

for (let i = 0; i < 10000000; i++) {
  workout_empty[i] = "push-up";
}

for (let i = 0; i < 10000000; i++) {
  workout_slot_ready[i] = "push-up";
}

The rule of thumb is that pushing that many elements to a completely empty array will be very slow. JavaScript generally prefers when the array is already populated^[2], but let's actually test it.

The results exceed expectations. Pushing into the workout_slot_ready array is more than 23x faster!

Slot_ready	Empty
🏆 93 ops/s	4.1 ops/s

Quickest way to create filled array

So now we know that it's better to work with filled arrays, but how to create them quick? To find out let's set up the octagon for a fight between four opponents:

old-style and cheerful Grandpa - Array[i],
well-known and widely respected - push() method,
new kid on the block but with a cool hairstyle - new Array().fill(),
the guy who wears underpants on the outside - Array.from().

Let's ask them to do 10 000 000 push-ups and see who will win in terms of performance.

// creating an empty array and filling it ol' style!
const workout = [];
for (let i = 0; i < 10000000; i++) {
  workout[i] = "slot";
}

// creating an empty array and filling it up using a push
workout = [];
for (let i = 0; i < 10000000; i++) {
  workout.push("slot");
}

// using Array's fill method to fill the array
workout = new Array(10000000).fill("slot", 0, 10000000);
//      = new Array(10000000) performs exactly the same

// using Array's static from method with an object literal
workout = Array.from({ length: 10000000 }, () => "slot");

The results: new Array().fill is 4.8x faster than push and then almost 14x faster than Array.from() creep!

new Array().fill	push()	Array[i]	Array.from()
🏆 18 ops/s	3.7 ops/s	3.5 ops/s	1.3 ops/s

Quickest way to insert elements into array

So let's assume that our workout array is already blazingly-fast-ninja-style-filled^🥷🏻⚡.

Now let's push some push-ups into the workout! By doing so, we need to approach a well-know JavaScript battleground of fight between forEach and for ⚔️.

// imperative approach
for (let i = 0; i < 10000000; i++) {
  workout[i] = "push-up";
}

// declarative approach
workout.forEach((_, i) => workout[i] = "push-up");

This might cause some declarative devs to cry a little in the corner, but face the truth! Your fighting techniques are lame-o. Imperative 4 life, with performance 12.8x faster than declarative.

for	forEach
🏆 86 ops/s	6.7 ops/s

Final comparison

Let's lay the groundwork for one final performance battle: the laydev technique from the beginning of our training and a ninja-style crafted method, versus a whole bunch of push-ups 💪🏻!

// 🐌 laydev style
const workout = [];

for (let i = 0; i < 10000000; i++) {
  workout.push("push-up");
}

// 🥷🏻 ninja style
workout = new Array(10000000).fill("slot", 0, 10000000);

for (let i = 0; i < 10000000; i++) {
  workout[i] = "push-up";
}

The results are just awesome - we are 4.2x faster!^[3] 🎉

Ninja styla	Lay style
🏆 17 ops/s	4 ops/s

Tip

Next time you set yourself up for a good ninja push-up training, leave that fancy declarative kimono at home. Dress properly in ninja-imperative shinobi shōzoku outfit and remember to warm-up your workout array with some fill method^[4] and use of good, ol' for loop.

RIP in peace Akira Toriyama. ↩︎
If you are keen to learn why is it like this check this blog. ↩︎
It's worth mentioning that those results were measured using V8 browser (which is currently covering 75.5% of the market). SpiderMonkey actually makes ninja style slower than laydev (with 3.04% of the market). ↩︎
Actually clean new Array(10000000) performs excatly as efficient as the one with fill, so this step could be potentially skipped (although this way we might be creating holes in the array). ↩︎

Passing custom class object to web workers

2024-03-18T00:00:00Z

Annoy level	Time wasted^[1]	Solvable
	6h	Kinda

Web workers are powerful sorcery in the battle against performance demons. When utilizing them, everyone gains a +10^🛡️ defense bonus against senior devs and a +5^🐎 boost to development stamina.

However, like every sword, they have two edges. One of these edges, hidden in the depths of MDN, can become a real pain in the ass when attempting to pass something more complex than an ordinary Object^[2].

Let's take a look at the simple planWorker.ts code:

onmessage = function (e: MessageEvent): void {
  const x = e.data.x as number;
  const y = e.data.y as number;

  const layer = new Layer(x, y); // layer type is Layer

  postMessage({ layer: layer });
};

The worker gathers the data and then creates a new Layer with the given arguments. After the work is done, the new layer is sent back and can be accessed as e.data.layer.

Now, let's examine the process method of the Frame class that created this web worker in attempt to push the returned object named layer into the layers property in its class:

export default class Frame {
  layers: Layer[] = [];

  public process({ x, y }: Coords): void {
    const worker = new Worker(
      new URL("../utils/planWorker.ts", import.meta.url)
    );

    worker.onmessage = (e: MessageEvent) => {
      const layer: Layer = e.data.layer; // layer type is Layer, right?

      this.layers.push(layer);
      worker.terminate();
    };

    worker.postMessage({
      x: x,
      y: y,
    });
  }
}

How pleasantly surprising it was to discover that suddenly methods working on the layers array couldn't access some of the methods from the Layer class. Well I think that means it's...

After many hours of debugging, I discovered this GitHub question and realized that, unlike every other time, the fault was not on my side.

The issue lay within the web worker messaging system.

Caution

If your custom class contains something more then just properties then buckle your seatbelt Dorothy, 'cause they are going bye-bye!^[3]. The structured cloning algorithm used by web worker message system will ignore all of yours:

property descriptors
decorators
setters,
getters,
methods

The solution to handle this ✨feature✨ is to either construct new object properties to maintain the current state of the original class or utilize JSON. However, the loss of methods may be unacceptable for some, thus making web workers less than ideal for handling more complex custom class objects^[4].

I learn something for sure, but you know... I could spend that time more productive, like watching Adventure Time instead. ↩︎
Web workers can also pass Array, ArrayBuffer, Boolean, DataView, Date, Map, Number, Primitive types, except symbol, RegExp, Set, String and TypedArray. ↩︎
Reference to Matrix movie. ↩︎
Other constrain may be the fact that all objects are copied not shared when used by messages system. Sharing data is possible, but not with the objects. ↩︎

AnnoyScript

Adding view count and like button to 11ty

Cooking the solution #

Web workers with React and TypeScript

The good 🦸‍♂️ #

The bad 🦹‍♂️ #

The ugly 🥸 #

Further optimalization #