PeterGoes.nl Blog

Updates for 02-2023

2023-03-04T20:03:29Z

## Personal stuff

### The road to 7a (bouldering)

I really want to nail the 7a this year. Turns out, my bouldering gym offers personal training. 3 sessions of an hour. After 2 sessions, I am even more excited. It sure is more difficult to climb when you have to think about what you do, but I feel I am making progress. 10/10 would recommend.

### Going to concerts again!

Having kids (6 and 3 years old) makes sure that time is scarce. I thought that going to concerts was a thing of the past, especially after the pandemic. I am not subscribed to newsletters of venues anymore, so I miss quite a lot of opportunities as well. A colleague of mine does not. Lucky me!

After a couple of concerts now, I could not be more happy to stand in a closed space with to much people around listening to really hard music again! Man, did I missed this!

## Work stuff

### Frontend Masters subscription

I decided to sign up for Frontend Masters with my education budget this year. It is awesome. The learning paths that they offer are great and the content is of such a high level.

I was looking for good courses specifically on CSS. Great material is out there for JavaScript, but I really want to step up my CSS game. Although Frontend Masters is more JavaScript focussed, there are also quite some CSS courses. When I asked about good courses on Mastodon recently, I got some other good examples, so I will spend a good amount of time learning things the coming year!

### Mixed feelings about the ranting on React

Ranting about React and JavaScript frameworks is in full swing I my social network circles. While I consider myself on the progressive enhancement / use as little JS as possible side of The Divide, I do use Preact daily at work.

I feel personally attacked by all the ranting and Keith J Grant formulated my feelings exactly in his post React Pays the Bills

That is to say, now that an opportunity has presented itself to reevaluate our current Front-end stack. I am thinking hard about what a framework-less front-end setup of a large website would look like.

Looking at user stats increase my emotional connection with them

2023-03-04T14:06:42Z

It seems that the capabilities of CSS grow exponentially fast the last few years. With the iterop initiative making sure that features are consistent across browsers, life as a Front-end Dev is quite awesome.

Usually when I want to play with all these new toys, I look at Can I Use to see if I can actually use them. But now that we are starting to build a complete design system from scratch, I want to know if I could use features like container queries or the :has() selector. It could, potentially, reduce the complexity of our components quite drastically.

Looking at statistics from our actual users

I keep a close eye to the statistics from our analytics tool to see what browsers our readers use. However, we are very privacy conscious. We use a privacy friendly analytics tool (Matomo), respect the do not track header, track only the data we actually need. That does mean, however, that we don’t know a lot about our users. The section ‘unknown’ in the browser engine section is quite large.

To actually make more informed decisions, I wrote a small script that test CSS features I care about and track those.
Now I have actual data from our users. But then I noticed something when I looked at the data.

I care much more about stats from our actual users than nation wide stats

When I was primarily looking at Can I Use, there was no emotional connection. 85% of users did have Grid support? Great, start using it. Maybe not on critical things, so those 15% that do not have it do not suffer that much.

But now, I have stats like: 91% of our members browsers support the :has() selector. That means 9% of our members, which pay my salary, do not. All of a sudden that is quite a substantial amount!

I find it an even more exiting challenge to provide a base line which I can then progressively enhance. Because these stats remind me that our users are people, not a number.

Ditching the car turned out to be a very good idea

2023-01-29T17:05:01Z

We just hit 1500km on our cargo bike. After our car broke down in the summer of 2022 we decided to not replace it. Instead we got an electric cargo bike and a subscription for a car share service. It still felt like a challenge with two kids (2 and 5 years old at the time).

After just two months since we got the cargo bike, we reached the first 1000km (December 24 2022). Today we reached 1500km.

The car share subscription feels like a cheat code. I don’t like to rent a car through it, but we do so from time to time.
We do use the cargo bike every single day though. That feels great. We started out with the environment in mind, reducing our footprint. But the real benefit comes from some place else.

We have so much more quality time with each other as a family. It takes longer to travel, sure. We have to plan ahead and we can not do everything we did when we had a car before. In the weekends, we used to go out quite a lot. Now that we have to calculate bike time, we are limited in what we are able to do. But we gain time with each other.

The conversations we have when the kids are asleep in the bike are on another level from those we had when we sat in a car. Being outside does something with your mind. You are more relaxed.

But when the kids are awake, they get to explore the world around them much better. Because the bike is slower, the can see and hear much more.

I was a bit skeptical at first for the winter period. The dutch weather is quite wet and not all that pleasant to ride in. But the way it turns out? I can not wait until summer.

If the previous months are any indication, the summer of 2023 will be amazing.

A few updates for 01-2023

2023-01-27T19:50:21Z

Maybe a summary of the previous month is a good way to get in the habit of blogging more? Lets find out…

## Personal stuff

### The road to a 7a (bouldering)

I don’t make new year resolutions. I am notoriously bad in keeping those kind of promises to myself. But this year I did set a goal for myself. Taking bouldering really serious again and being able to climb a 7a (V6 on V-scale?). It is an ambitious goal for sure, but I really like it!

Yesterday I did my first 6b, so progress is being made.

### Struggling with my own website

I really like having my own website. I like the promise of having a true place of my own that I can experiment with. But I find myself in a place where I want to have a proper looking website because I am a front-end developer. Having a broken website looks bad I guess? That fear of having a broken website and what people might think of me as a front-end dev as a result prevents me from having that hacker mindset. The thing that makes having a personal website fun in the first place.

The same goes for blogging, I am really happy that I found the RSS Club, because this post is not ‘published’, it makes it safer to fool around. I want that same feeling again for the site as a whole.

On a related note, I am really in the market for a good combination of writing markdown with a tool like IA Writer and One Click Publish convenience of a CMS…

## Work related stuff

### Arc Browser

Since a couple of weeks I switched to Arc Browser. And MAN, does that thing live up to the hype! I am SO happy to have a browser that is primarily keyboard driven. I almost never have the side panel open so I am looking at each page in all of its glory in an almost full screen view. I absolutely love it. And Split Mode is awesome!

### Learning with Remix

I am in the process of rebuilding the whole front-end of our codebase at De Correspondent with Remix. At first I was sceptical about a React driven framework, but the focus on foundational web tech is amazing. I truly think I am becoming a better front-end developer just by using this framework.

For example, yesterday I needed to throw an Error. It did not behave exactly as I expected, so I asked around in the discord server, and someone pointed me to the MDN page of the Error object. I learned a thing or two about the cause option.

Love it.

I got genuenly exited to find out about a secret society!

2023-01-10T21:41:23Z

I love the web. The openness, the expressiveness. The way it lets me fiddle with code and make things come alive. After the Twitter collapse I joined Mastodon, and the whole front-end community with me (or so it seems). With that I regaind an appreciation for blogs and RSS. So when I found out about this, I could not be happier. It feels like the old days of the web in a sense that it you get to access things that other people don’t know about, while working out in the open. It is weird to describe, I love to make stuff available for an as wide as possible audience, but this is somehow different? I don’t know.

I make myself the promise (again, I know) to pick up blogging. Only this time to allow for more personal things as well. Maybe that makes it more easy. So to kick that off: I promised myself I can boulder a 7a by the end of 2023. Is that a reasonable goal? I don’t know, we’ll see :). I will keep you posted!

The Set-Cookie and Cookie headers got me confused, but I think I get it now

2022-12-14T19:56:05Z

I learned a thing or two about cookies today… It took me quite a while to figure it out, but I think I understand it now.

I am rebuilding a website from an old PHP / Twig setup into something more modern. The PHP backend is still in use, but I wanted to separate the front-end and rebuild that with Remix. The backend gets slowly transferred into an API only backend that my Remix server is going to communicate with.

Authentication is done via cookies, but since I now have a Remix server between the client and the API, I need to perform the requests to the backend with the session cookies from the user. Having the Remix server in between allows me to do a couple of requests to the API without going all the way back to the client for each response. But that means getting and sending cookies along as well.

The problem I faced was this: When I create a user, I also need to store the country that user lives in. For the user, that is one form to will out: username, password, countryCode. But the backend wants to store the countryCode via a separate request. So first create the user, then update that user via an other endpoint with the country code. But only an authenticated user can update their country code.

I needed to get the session cookie from the createUser endpoint, and provide that to the updateCountry endpoint right after.

`Set-Cookie` and `Cookie` are two different things

It took me a while before I realised that the server sends a Set-Cookie header to the browser, but expects a Cookie header when receiving.

When I figured that out, it was a matter of getting the Set-Cookie header, and passing it as Cookie, right? No, that did not work at all.

Looking at the original requests from the legacy codebase, each request to the backend has a Cookie request header with some information (session=s3ss10n1d for example). But multiple cookies can be stored for a domain, so the value can also be: session=s3ss10n1d; setting=theme-blue. Right, a header, cookies split via ; . I get that.

Looking at the response headers though, there were multiple Set-Cookie headers. One for each cookie, each of that with the following shape: cookieName=cookieValue; expires=Thu, 01-Jan-1970 00:00:01 GMT; Max-Age=0; path=/; domain=.some-site.com; secure; HttpOnly; SameSite=Lax. Ok, the cookie has a name / value combo and some attributes separated by a ; . I get that as well.

Reading response headers

The Response contains a headers property that is an instance of Header. You can use it to query values of response headers via header.get(). From the MDN documentation:

If the header has multiple values associated with it, the byte string will contain all the values, in the order they were added to the Headers object

The code example with it:

myHeaders.get('Accept-Encoding'); // Returns "deflate, gzip"
myHeaders.get('Accept-Encoding').split(',').map((v) => v.trimStart()); // Returns [ "deflate", "gzip" ]

That does not seem so hard… Let’s have one more close look at the Set-Cookie value: cookieName=cookieValue; expires=Thu, 01-Jan-1970 00:00:01 GMT; Max-Age=0; path=/; domain=.some-site.com; secure; HttpOnly; SameSite=Lax. Please notice the expires value: Thu, 01-jan.... That , right there is the sole reason a good amount of hours got wasted. There is no way to tell where to split the string without intimate knowledge of all possible attributes for the cookie…

Parsing the Set-Cookie header with a library finally solved the problem I had.

Lessons learned

A server sends multiple Set-Cookie headers where each header contains a cookie with additional attributes, split by ; .
A server expects a single Cookie header where each cookie is represented via name=value split by a ; .
For the love of all that is dear to you, use some libraries for dealing with the Set-Cookie header. I ended up using cookie and (more importantly) set-cookie-parser.

Should a website provide configurable accessibility options to users?

2021-06-16T09:33:35Z

Update about the used example in this post

Today (17th of June 2021) I stumbled upon a CSS Tricks article which states that you CAN detect if a user has their colours inverted system wide. Kicking over the whole foundation of the examples I provide in this post. The original problem that I had is thus solved (use the @media (inverted-colors) media query). If you still want to read the original post, let’s get going.

Today I did an interview for my accessibility research (in Dutch, Chrome translates it well). The person I spoke with used the system setting to invert all colours, due to hyper sensitivity to light. The result was that images were also inverted and thus became negative:

Here you can compare how inverted colours look on macOS. The left side shows a Google Images search without inverted colours. In the right side the colours are inverted. The images then become negatives. Making them really hard to interpret.

She uses social media on a regular basis. Seeing all images and videos in their negative form, takes the fun out of using it. When an image or video contains crucial information, it becomes a bigger problem.

On iOS there is the option to “smart invert” the colours. Meaning, all colours are inverted, except for images and video. That is the golden solution in this scenario, but I did not find similar “smart” colour invert options in macOS, Android or Windows.

As a developer, I can provide an option to invert the colours of an image first, and when the operating system inverts all the colours on screen, my inverted image is again inverted, making it appear normal again. Like this:

This time, an invert() filter is applied within the website itself. Again, the left side shows a ‘normal’ view but with inverted colours. The right side shows the inverted version. But because the images were already inverted, they appear normal again.

You can’t (rightfully) detect if accessibility features are enabled

The developer of a website can not find out if certain accessibility features are turned on or not. Not only are these settings handled system wide, from a privacy perspective a user might have good reasons not to disclose to every website they required these options.

If I want to allow the users to browse the site with inverted images, so that the operating system can reinvent them, I need to provide a UI for that. A accessibility settings menu if you will. That in turn requires the user to a) find it, and b) actively turn it on.

Questions to answer

All this considered, I have a couple of questions to answer:

Do users want this level of customisation?

I get fed up with all the cookie consent notices for every single website. I can imagine someone does not want to search for settings on every single website they visit.

But for an app or a site you use every day, you might want to go through that trouble.

Looking at the Overlay Factsheet, other attempts to ‘fix accessibility’ are not that well received. Although I don’t think this menu is comparable to these tools, It is vital that a user understands that as well.

How do I make sure that this does not become a tracking data point?

You don’t want to reapply the same setting every time you visit a site or app. Persisting the setting for the user is important. But that will (potentially) allow others to find out these settings have been set. Which violates the users privacy.

Do you have any answers?

Did I overlook important details? Are there existing solutions out there? Do you have an other opinion? I like to know about it!

Review incoming WebMentions before publishing with Github Actions

2021-05-26T19:10:17Z

Recently I added webmentions to this blog.
Which is completely static. It is build with Eleventy,
and hosted on Netlify.

Accepting Webmentions in the first place

Accepting webmentions for a static site is relatively straight forward. I
followed the article An In-Depth Tutorial of Webmentions + Eleventy
to do so. And was quite happy with the results.

But there are a few things I did not liked about it:

I want incomming mentions to be published as soon as possible, without the
need for client-side JavaScript. Something needs to trigger the build.
I want to be in full control of the content placed on this site. Accepting
webmentions is awesome, but I want to be able moderate them.

Using a Github Action to create a Pull Request with new mentions

I use webmention.io to receive my mentions. The service
comes with an API with which I can retrieve them.

I created a Github Action to periodically fetch my mentions and write each of
them to a file.

When the Github Action runs, it should:

Fetch my web mentions from the API
Store each of them in a seperate json file
Create a Pull Request with the new and changed files

Fetching and storing new Web Mentions

The (trimmed down version of the) fetching and storing logic comes down to this:

const mentionsFolder = './mentions'

fetch(`https://webmention.io/api/mentions.jf2?token=<MY-TOKEN>&per-page=10000`)
  .then(response => response.json())
  .then(data => data.children)
  .then(mentions => mentions.forEach(writeMentionToFile))

function writeMentionToFile(mention) {
  const id = mention['wm-id']
  const targetPostPath = mention['wm-target'].replace('https://www.petergoes.nl', '')
  const outputFolder = path.join(mentionsFolder, targetPostPath)

  fs.writeFileSync(
    path.join(outputFolder, `${id}.json`),
    JSON.stringify(mention, null, 2),
    { encoding: 'utf-8' }
  )
}

I left out some checks here and there.
This is the full store-webmentions.mjs
file.

Any webmention is stored as a json file in the mentions/ folder. If a mention exists, it gets overwritten with the new content. Updating
that mention.

The Github Action

Lets look at the actions .yml file.

name: Webmentions

on:
  schedule:
    - cron: "0 0 * * *"

jobs:
  webmentions:
    steps:
      - name: Check out repository
        uses: actions/checkout@master

      - name: Set up Node.js
        uses: actions/setup-node@master

      - name: Install dependencies
        run: npm install

      - name: Fetch webmentions
        env:
          WEBMENTION_IO_TOKEN: <<WEBMENTION_IO_TOKEN>>
        run: node ./_automations/store-webmentions.mjs

      - name: Create Pull Request
        id: cpr
        uses: peter-evans/create-pull-request@v3
        with:
          token: <<GITHUB_TOKEN>>
          commit-message: Update WebMentions
          assignees: petergoes
          title: Update Webmentions
          body: Update Webmentions

This is the full webmentions.yml file.

I set the schedule to run the action every day at midnight. New mentions are
automatically pulled in at least once a day.

When there are new mentions, the Create Pull Request
action commits these changes and generates a Pull Request. If there are no new
mentions, it does nothing.

This works fine, but we can take it one step further.

Run the Github Action when a new mention is received

Instead of doing this once a day, it would be nice if this action runs anytime
a new mention is posted. The webmention.io service provides the option to call a
webhook when a new mention is posted. Nice!

Github Actions can be configured so that they are triggerd via a webhook. Sounds like
a match made in heaven.

To set this up, I followed the guide GitHub Actions Trigger Via Webhooks.

Lets first configure the Github Action to run when a webhook is called:

 on:
+  repository_dispatch:
+    types: [Received Webmention]
   schedule:
     - cron: "0 0 * * *"

To run the action, I need to make a POST request to: https://api.github.com/repos/petergoes/petergoes.nl/dispatches.
The body of the request needs to contain a type I defined in the .yml file, and
I have to include a token in the Authorization header.

The full request looks like this:

fetch(
  'https://api.github.com/repos/petergoes/petergoes.nl/dispatches',
  {
    method: 'POST',
    headers: {
      Authorization: `token <GITHUB_TOKEN>`
    },
    body: '{ "event_type": "Received Webmention" }'
  }
)

That does not match the data structure I get from the webmentions.io webhook…

Use a Netlify function to call the Github Action

The last piece of the puzzel is a Netlify function which receives the webhook
request from webmention.io, when that is valid, it calls the endpoint on Github
to trigger the Github Action

The Netlify function looks like this:

const fetch = require('node-fetch')

exports.handler = async function (event, context) {

  // The payload from webmention.io is configured to contain a secret
  const { secret } = JSON.parse(event.body)

  // Make sure that it is known to the Netlify Function so no one else can call
  // this function
  if (secret !== process.env.WEBMENTION_IO_SECRET) {
    return { statusCode: 401 }
  }

  // Trigger the Github Action
  const response = await fetch(
    'https://api.github.com/repos/petergoes/petergoes.nl/dispatches',
    {
      method: 'POST',
      headers: {
        Authorization: `token <GITHUB_TOKEN>`
      },
      body: '{ "event_type": "Received Webmention" }'
    }
  )

  return { statusCode: 200, body: 'ok' }
}

See the full Netlify Function here

And thats it! That are a lot of steps. But when you post a webmention, or like the
tweet on Twitter, you should see a PR in the Pull Request section on GitHub
in a few minutes!

With great power comes great responsibility

2021-05-15T10:00:39Z

When you build a UI for a product (app, website, anything else), it is intended to be used by people. People come in all shapes, sizes and abilities. I used to think that making an UI accessible is something you should aim for. It was morally the right thing to do.

Recently I started out to become better at making my work better accessible by doing research into people that use assistive technology. You can follow along with my research here (in Dutch for now). In the first interview, it became clear to me.

We have the power to influence other peoples lives

When building a product, we allow people to do their jobs. We build the interface through which others do their work.

We can add a feature that makes someone perform a task more efficiently. The less time it takes to use our UI, the more effective that person can be in their job.

But the other way around is equally true. If we make it harder for someone to use our product, we make them less effective.

Accessibility is not a nice to have, it is your responsibility

Making your UI accessible, is allowing someone who relies on assistive technology to work at all.

By not taking care of accessibility, a team takes away the self-sustainability of a person. It prevents them from having a job.

We as designers/developers have the power to make, or break, someones career. Thats a lot of power, and an equal amount of responsibility.

Binding behaviour to HTML with Web Components as Progressive Enhancement

2021-04-28T17:48:36Z

For a couple of years now, I have been working with front-end frameworks almost exclusively. At De Voorhoede, we adopted the componentised way of working early on. We moved from binding JavaScript to the DOM via data-* attributes on to using Angular.JS and from there to React and Vue.

Each step of the way felt like a great step forwards. The concept of components spoke to me because I could focus on a single feature and have all relevant parts close by. The HTML, CSS and JavaScript moved closer and closer together, making it easier to work on isolated features / UI elements. The tooling became better and better. For the bigger projects that we do, this way of working fits us good.

But the downside is that it requires a lot of build steps, and results in pretty heavy JavaScript bundles.

For smaller (or simpeler) sites, that do not require a lot of JavaScript (like this website), a large Front-end framework is overkill. The question becomes again: “How do you manage JavaScript when you use a HTML templating language?”

The folks over at Grrr (checkout Grrr’s tech blog!), have released
Hansel to deal with this. I have used that for a couple of projects, and I like the philosophy behind it. But for this website, I wanted a build step free developer experience. By using a small set of features of the Custom Elements spec, I think I hit the sweet spot.

Componentised UI with a HTML templating engine

To explain the concept, I’ll use the tag-list component from the /bookmarks page. On this page, I’ll store bookmarks, which have tags on them. The tags are not yet displayed when I write this though.

Besides the big list of all bookmarks, I wanted a way to filter them by tag. Eleventy gave me a hard time in doing this during build time. Because of that, I wanted to do it with JavaScript.

First, lets look at the template side:

<my-tag-list class="tag-list">
  <details>
    <summary>All tags ({{ tags.length }})</summary>
    <ul class="tag-list__list">
      {% for tag in tags %}
        <li class="tag-list__item">
          <a
             class="tag"
             href="/bookmarks/tags/{{ tag.slug | slug }}/"
          >
          {{ tag.label }}
          </a>
        </li>
      {% endfor %}
    </ul>
  </details>
</my-tag-list>

A details/summary combination is used to show / hide the tags. Nunjucks templating is used to render the tags. For each tag, a link to an overview page of that tag is rendered.

But to see anything when you clink on that link, you’ll have to have JavaScript enabled. Not only enabled, but it should be run in a browser which can run ES6. Because the page /bookmarks/tags/:tagName requirers it.

I want to hide the whole tag list from users that do not have JavaScript enabled, but display it to users that do.

The CSS for this logic looks like this:

.tag-list {
  min-height: var(--line-height);
}

.tag-list details {
  display: none;
}

.tag-list--show details {
  display: initial;
}

First I define the .tag-list class. It gets a min-height because I want to reserve space for the summary when it will appear. That way, it does reflow the page when the component is loaded.

Next, I hide the whole details element. Users without JavaScript will not know it’s there.

But then I define the .tag-list--show selector which resets the hiding of the details element. I need to apply that class with JavaScript.

This is how I do that with JavaScript:

class MyTagList extends HTMLElement {
  connectedCallback() {
    this.classList.add('tag-list--show')
  }
}

customElements.define('my-tag-list', MyTagList)

Here I define the Custom Element MyTagList. When the element is mounted into the DOM, the connectedCallback fires. Because this is a Custom Element, the this refers to the instance of my-tag-list in the DOM. Within the connectedCallback I can do as much DOM manipulation as I please. It is guaranteed that all the DOM inside the my-tag-list element is ready.

Using a Custom Element here provides me with a couple of benefits:

Benefits

I don’t have to write the boilerplate code of querySelect-ing elements matching a data attribute and triggering functions
When elements are dynamically added to the page, I don’t have to rerun that boilerplate code
By using Custom Elements, I need to provide the type="module" attribute when I load the script. This means, that older browsers do not run that code at all. I don’t have to worry about transpiling my code back to ES5.

Closing thoughts

This is a small example, but I hope it illustrates the use case. I am sure there are downsides to this method but, for now, I like it and want to try it out on more projects.

What do you think? Did I mis anything?

Why I signed the Overlay Factsheet

2021-03-13T00:00:00Z

Today I submitted the PR
to sign the Overlay Fact Sheet. I did not do
this lightly. In this post, I explain why I did.

What the factsheet is all about

To quote the conclusion of the Overlay Factsheet:

Accessibility on the Web is a big challenge, both for owners of websites and
for the users of those websites. The invention of novel approaches to
resolving this challenge is to be commended. However, in the case of overlays

especially those which attempt to add widgets that present assistive
features - the challenge is not being met. Even more problematic are the
deceptive marketing provided by some overlay vendors who promise that
implementing their product will give their customer’s sites immediate
compliance with laws and standards.

No overlay product on the market can cause a website to become fully compliant
with any existing accessibility standard and therefore cannot eliminate legal
risk.

overlayfactsheet.com

Why I signed

I view front-end development as a craft. I take pride in code that is readable and maintainable, in interfaces that make sense to its users. I try my best to become good in what I do. It takes commitment and dedication to reach that. And like with anything that takes dedication to master, a good front-end is not cheap.

Front-end development is not a matter of ticking boxes, or putting colour onto various elements. You should be able to look past your own experiences, think about someone else; the user of your work, the browser that user is using and the context in which the user is operating.

A part of the code that we write can be automatically checked and fixed. But for a pretty big portion that is not an option. It is the human part of our work. Our ability to research how our users interact with our product, translate that into a good and accessible user interface, can never be automated. Because making your product accessible, comes down to building it so that it makes sense to your user. That is, by definition, subjective. And computers can not make choices in subjective matters.

Therefore, tools that promise to make a website accessible with a single line of code, can never fix issues that require human thinking.

When they do make that promise, they false advertise.
And degrading the craft of front-end development with it.

Learn about Web Components CSS selectors by server side rendering

2019-04-27T00:00:00Z

New CSS selectors in Web Components

One of the most useful features of the Shadow DOM specification, is scoped CSS. The styles defined in the Shadow DOM don’t leak out to the containing page. But within the Shadow DOM you CAN get a sense of the surrounding context of the Custom Element. To make the latter possible, some new selectors have been introduced. In this post, I will explain how I learned about some of these new selectors, focusing on the :host(), :host-context() and ::slotted().

In a previous blog posts I focussed on serialising the HTML of the Shadow DOM. But these new CSS selectors should also be transformed and placed in the main document. Because, as long as the Web Component did not load on the client yet, its styles will not show up.

So somehow I needed to transform the selectors from the Shadow DOM styles and move these styles into the main document (or Light DOM). That process taught me quite a lot about how these new selectors work!

A quick recap of serialising the HTML

To serialise the Shadow DOM I go through this process:

Launch Puppeteer and load a page containing Web Components
For every Web Component:
1. Find the rendered Shadow DOM
2. Find the Light DOM contained in the element
3. Move the Light DOM into a <template> element
4. Move the (rendered) Shadow DOM in the place of the Light DOM
5. Add a couple of attributes indicating the state of the serialising process
Add a rehydrate script to the page which will run on the client undoing all the work above

Differentiate SSR states in a Web Component

2019-04-15T00:00:00Z

The problem

After my previous blog post, I ended up in a discussion with @harmenjanssen in which he made the following point:

Most components will need Javascript to function, *after* needing Javascript to render, is what I mean. So using SSR to implement the JS *rendering* would leave the user in a sort of semi-enhanced experience?
At least I need some examples that show otherwise.
— Harmen Janssen (@harmenjanssen) March 25, 2019

This got me thinking. I am used to writing Vue components. I use Nuxt to server side render my applications and I did not encounter much issues like Harmen described.

Sure, we have created a <no-ssr> Vue component at De Voorhoede, but generally speaking, I don’t think about it all that much. Most of the time it Just Works™️.

To understand the point of view of Harmen, let’s dive into a blog post he wrote recently. I encourage you to read the post, it’s really good. But the gist of it is this. By using the enhancer pattern you enhance a piece of html with JavaScript to make it interactive.

I don’t know the stack Harmen uses, but my guess would be that a cms outputs html. The client side (vanilla) JavaScript then kicks in and enhances that html. My assumption is that the cms is not using a front-end framework like Vue or React to build that html. It is a vastly different approach to outputting html via a front-end framework like Vue or React where the same JavaScript runs both on the server and client (and JavaScript is the main language to write the component in).

In my attempt to server side render Web Components, the process is like this:

Write the component
Fire up a puppeteer instance and enhance it via server side JavaScript
Send the rendered component to the client
On the client, the component gets hydrated letting client side JavaScript take over

If you compare this flow to the workflow of enhancing html client side, my process enhances the component too early. As long as the client does not have the JavaScript downloaded, the components html is enhanced, but does not have JavaScript yet to back it up. That is the “semi-enhanced experience” Harmen is talking about.

Another thing that is quite different compared to the enhancer pattern, is that by using Vue or React, JavaScript is your templating language. Where on a typical server rendered site, the html would be written in php, nunjucks or something else. In that case JavaScript is only run client side and is only used to add behaviour. In the way I try to render a Web Component server side, I also use it as the templating language to create the html.

How does a front-end framework like Vue handle the issue of the “semi-enhanced experience”? Why did I not encounter this issue more often before?

Life cycle methods in Vue

I take Vue as an example here since I have the most experience with it, but the concepts apply to React as well.

A Vue component has the concept of life cycle methods. During various stages of its life, various methods are run. Let’s look at the mounted() method.
The documentation of the mounted method tells us:

Called after the instance has been mounted

So, when the component is mounted in the DOM, this method is called. But a little further down, the documentation states:

This hook is not called during server-side rendering.

So the mounted method only runs client side. This gives you the ability to make a difference between what is rendered server side, and what happens client side.

Further more, since Vue handles its components server side as well as client side, it can treat them differently based on the context.

When compared to the enhancer pattern, there are quite a lot of states a component can be in, in Vue. To name a few:

before-anything-happens server side
rendered server side
rendered server side on the client
mounted on the client

Looking at the enhancer pattern there are only two states:

non enhanced client side
enhanced client side

As it turns out, I did encounter the issue before, but because Vue as a framework makes working with components both on server and client side so straight forward, I do not have to think about the differences between server and client environments. That is kind of the whole reason to go for a universal JavaScript approach in the first place.

In my attempt to render Web Components server side, I did not take the work Vue does here into account. Because I don’t treat server and client environments as two separate environments, I actually spin up a client IN the server to render the output.

Life cycle hooks in Web Components

But wait a minute, Web Components have life cycle hooks of their own. They are called Custom Element Reactions. There is a constructor and a connectedCallback among others. The connectedCallback maps to Vues mounted pretty close. From the documentation:

Called every time the element is inserted into the DOM

That does not help me all that much though. I am still using an actual client when rendering server side. So although the mounted hook is not called server side in Vue. connectedCallback does get called in my implementation.

sigh…

So what are the options now? I really want this server side render thing I am building to not introduce any new api by creating some sort of base class which you have to extend in order to get server side rendering working. It should work on any Web Component.

Using what I already have

But then I looked at what I am already doing to the component and realised, I have the answer right here. In fact. I already wrote it!

When I serialise a component server side, I include an attribute to the element: data-ssr="serialized". When the hydration script takes over that attribute is updated to: data-ssr="hydrated". So now I do know which state my component is in!

Lets look at an example:

class LifeCycle extends HTMLElement {
    static get observedAttributes() {
        return ['data-ssr'];
      }

    connectedCallback() {
        this.attachShadow({ mode: 'open' });
        this.shadowRoot.innerHTML = `
            <h1>I am server side rendered</h1>
        `
    }

    attributeChangedCallback(name, oldValue, newValue) {
        if (name === 'data-ssr' && newValue === 'hydrated') {
            this.shadowRoot.querySelector('h1').innerText = 'I am client side renderd'
        }
    }
}
window.customElements.define('x-life-cycle', LifeCycle);

What happens here: I listen for changes to the data-ssr attribute. When it does change I know I am on the client because the hydrate script sets it to hydrated.

Now I can update the component like I would in a mounted method in a Vue component.

Closing thoughts

Is this an ideal solution? No far from it. The data-ssr attribute is something that I came up with, and it only works if you render the component server side with my tool. It is by no means standardised.

But, it gets the job done. And as long as there is no standardised way of server side rendering a Web Component, we will be tied to custom implementations like this one.

So, now that I got that out of my head, I am going to focus on serialising the css of the Web Component. That is something I am struggling with for quite some time now.

Do you have any comments? Please reach out at @petergoes

My stab at rendering Shadow Dom server side

2019-03-24T00:00:00Z

What is the Shadow Dom and why is it important?

The Shadow Dom is a specification that CanIUse summarises as:

Method of establishing and maintaining functional boundaries between DOM trees and how these trees interact with each other within a document, thus enabling better functional encapsulation within the DOM & CSS. - CanIUse.com

An element can have its own DOM which is hidden or inaccessible from the outside. Think of the <video> element. All controls; the play button, the volume controls, the play head, are all DOM elements. You can’t see or modify them because they live in the Shadow Dom of the element.
The Shadow Dom in a <video> element has a mode of 'closed'.

A Web Component that you write can also have its own Shadow Dom. As a component author you can set its mode to 'closed' (like the <video> element) or 'open'. When it’s 'open' you can inspect it in your DevTools or interact with it via JavaScript. It is important to note that the Shadow Dom gets created when the Web Component is connected, and that it is not part of the DOM that gets downloaded with rest of the HTML. In other words, when the browser downloads an HTML file, the Shadow Dom is not included.

Why do I want to render a Web Components Shadow Dom server side?

The kind of projects I typically do are static site generated websites. Which can be enhanced with JavaScript when that is loaded, but as much as all functionality should be usable without JavaScript.

If I approach Web Components the same way that I would with a Vue Component, most of the markup I write ends up in the Shadow Dom. And by default, that won’t get shipped with the HTML file. I would NEED the Web Component to be downloaded and parsed in order for the Shadow Dom to exist. That kind of goes against the idea of not relying on JavaScript.

Serialising the Shadow Dom

So, I need to come up with a way to transfer the Shadow Dom with the rest of the HTML. In my previous blog post (Experimenting with rendering Web Components) I explain that I took a lot of inspiration from @treshugart’s @skatejs/ssr.

Let me explain my approach, which is quite similar, but I can afford to use
Puppeteer because I pre-render all my pages during build time.

The problem

Lets look at the Shadow Dom of an <x-hello> Web Component:

<x-hello>
  World                  <!-- Light DOM -->
  #shadow-root(open)     <!-- Thing made up by DevTools -->
    <span>               <!-- Element in the Shadow DOM -->
      Hello
      <strong>
		    <slot>       <!-- Special element to host Light Dom -->
		      <#text>    <!-- reference to Light DOM -->
		    </slot>
		  </strong>
		  !
    </span>
</x-hello>

As you can see DevTools visualises the Shadow DOM, but that is there so you can inspect it. It does not actually exist.

As a quick reference, as a user of the <x-hello> element this is how it looks in the source code:

<x-hello>World</x-hello>

As you can see from the Shadow Dom example, the word World is between
<strong> tags, so it is displayed in bold. The word Hello is also within the Shadow DOM. It’s rendered, but when the Web Component has connected.

And like I said before, the Shadow DOM is not part of the HTML. So how do we let a non JavaScript browser know that it should render World in bold, or Hello at all?

Breaking down the problem

I have the luxury that I can use Puppeteer. Puppeteer is a headless Chrome browser, and Chrome happens to have support for Shadow DOM.

I can fire up a Puppeteer instance, load up my page, and then have it do things. Things like this:

Quick note, I will explain only the concepts here using simplified code. If you want to take a look at the actual code, checkout my progress thus far in the repo here. All references to the files in the repo, I will do at the current commit at the time of writing. It is not necessarily the most recent version of the file.

First I need to get access to the loaded page

const puppeteer = require('puppeteer')
const serialize = require('./lib/serialize')
const hydrate = require('./lib/hydrate')

(async (fileName) => {
  const browser = await puppeteer.launch()
  const page = await browser.newPage()
  
  page.on('load', async onLoad (...args) => {
    /* Do things with the loaded HTML */
    
    /* Get the modified HTML */
    
    /* Write the modified HTML to a new file */
  })

  await page.goto('file://' + path.join(__dirname, fileName))
})('/public/index.html')

Lets fill in the blanks here

1. Do things with the loaded HTML

This is the meat of the whole process. First lets get into the browser context

await page.$eval('html', serialize)

serialize is a module I required from the lib/ folder earlier and this is what it does (here you can find the whole file):

module.exports = function serialize (rootNode) {
  
  function serializeNode(node) {
    /* will explain later */
	} 
  
  [...rootNode.querySelectorAll('*')]
    .filter(element => /-/.test(element.nodeName))
    .forEach(serializeNode)
}

I get all the elements in the current rootNode (when it first runs, that would be <html>
I filter out all elements that do NOT have a - in their nodeName. Custom Elements are required to have a - in their tag name.
For each Custom Element I run the serializeNode function

** serializeNode: **

function serializeNode (node) {
  const lightDomNodes = node.childNodes
  const lightDomHtml = node.innerHTML
  const templateDom = document.createElement('template')
  const scriptData = document.createElement('script')
  const slot = node.shadowRoot.querySelector('slot')
  const attributesProperties = node.getAttributeNames()
    .filter(name => name !== 'data-ssr')
    .reduce((obj, name) => {
      return {...obj, [name]: node[name] }
    }, {})

  templateDom.setAttribute('type', 'ssr-light-dom')
  templateDom.innerHTML = lightDomHtml

  scriptData.setAttribute('type', 'ssr-data')
  scriptData.innerHTML = JSON.stringify(attributesProperties)
  
  // ...
}

Get a reference to the current Light DOM (elements as well as innerHTML)
Create template and script tags
Get a reference to the <slot> element
Get a JSON object with all attributes with their value

Then:

function serializeNode (node) {
  // ...

  /* move light nodes into shadowDom */
  lightDomNodes.forEach(lightNode => slot.parentNode.insertBefore(lightNode, slot))

  /* move shadowDom into root node */
  node.shadowRoot.childNodes.forEach(shadowNode => node.appendChild(shadowNode))

  /* remove slot element */
  if (slot) {
    slot.parentNode.removeChild(slot)
  }

  /* serialize custom element child nodes */
  serialize(node)

  /* add original lightDom as template */
  if (templateDom.innerHTML !== '') {
    node.appendChild(templateDom)
  }

  /* add the data as script */
  if (scriptData.innerHTML !== '{}') {
    node.appendChild(scriptData)
  }

  node.setAttribute('data-ssr', 'serialized')
}

Move Light DOM before the <slot> element of the Shadow DOM. The browser will also (sort of) move the Light DOM into the Shadow DOM. So let’s mimic that. I don’t place it inside of the <slot> because I will remove the <slot> later on
Move the (current) Shadow DOM (with the Light DOM added to it) into the Light DOM. We need to send the Shadow DOM over the wire. By placing its contents inside the Light DOM, its contents no longer are part of the Shadow DOM but become Light DOM
Remove the obsolete <slot> element
Trigger the same routine for the now-in-Light-DOM-elements as they might contain other Custom Elements
Add the original Light DOM in a <template> element. We need to preserve the original Light DOM because in the browser we should be able to reset everything to its original state
Add the (JSON) object containing all data as well to it to will gets send down the wire
Mark this Custom Element as 'serialized' so the hydrate script in the browser knows this elements needs some work.

I move the original Light DOM inside of a <template> element, because the browser won’t touch anything inside a <template>. In other words, it won’t render duplicate content.

Get the modified HTML

Now that I am done with modifying the HTML in the browser context, let’s get back in the Puppeteer script. I need the modified HTML:

page.on('load', async (...args) => {
  // ...
  const pageContent = await page.content()
  // ...
})

Now I have the modified HTML in the Puppeteer context

Write the modified HTML to a new file

Lets write the modified HTML to the target file

page.on('load', async (...args) => {
  // ...
  fs.writeFile(
    path.join(__dirname, fileName.replace('.html', '.ssr.html')),
    pageContent.replace('</body>', `${hydrate}</body>`),
    {encoding: 'utf8'},
    async (err) => {
      await browser.close();
    }
  )
})

Replace the current file extension to *.ssr.html so we know that this file is the modified version
Yet another slight mutation to the HTML: include the hydrate script.
When all is done, close the browser

So! Thats that! We now have a server side rendered version of all Web Components in a Light DOM fashion. Note that I did not do anything with CSS yet. I focused on serialising the HTML!! The CSS part is something I still have to figure out.

But the browser has all these capabilities. It would be a shame to not use them. Lets have a look at the hydrate script to see how the process can be reverted client side

Hydrate client side

function hydrate() {
  [...document.querySelectorAll('[data-ssr="serialized"]')]
    .forEach(el => {
      const lightDom = el.querySelector('[type="ssr-light-dom"]')
      const lightDomContent = lightDom.content
      const dataElement = el.querySelector('[type="ssr-data"]')
      const dataContent = dataElement.innerText
      const data = JSON.parse(dataContent)

      el.childNodes.forEach(node => {
        if (node !== lightDom) {
          node.parentElement.removeChild(node)
        }
      })

      lightDomContent.childNodes.forEach(node => el.appendChild(node))
      lightDom.parentElement.removeChild(lightDom)
      dataElement.parentElement.removeChild(dataElement)
      Object.keys(data).forEach(key => {
        el[key] = data[key]
      })

      el.setAttribute('data-ssr', 'hydrated')
      }
    )
}

module.exports = `<script>${hydrate.toString()}; hydrate();</script>`

That is the script at once. I simplified it a bit by removing some is-this-value-non-empty-checks. The original source is here This is going on:

Get all elements on the page with a [data-ssr="serialized"] attribute.
For each element do:
Get a reference to the <template> and <script> elements and their contents, in the current Web Component element
Remove all elements except the <template>. Removing all would-be-Shadow-DOM
Move all elements in the <template> element (original Light DOM) back into their original place
Remove the now empty <template> element and <script> element
Loop over the data object to add all properties to the Web Components properties. This makes sure Array and Object properties are set via JavaScript
Update the data-ssr attribute to 'hydrated' to indicate that this Web Component has been hydrated

Things I did not cover

As stated before, I only focused on serialising the HTML. CSS is something I did not cover at all. But even then, the HTML part is not fully complete yet. As you might know, Web Components can have multiple <slot> elements. The process thus far only covers a Web Component with a single <slot>!

And that’s all there is to it. I am sure I can improve this code quite a lot. But it gets me started.

If you want to have a look at the full code, check out the repo here.

Do you have any comments? Reach out to me at @petergoes!

Experimenting with rendering Web Components

2019-03-23T00:00:00Z

About this serie of posts

This is the start of my journey to render Web Components on the server. In it I will document the things I run in to. Things that work out. Things that do not work out. I see a great deal of potential in the adoption of web components because I believe that working with ‘vanilla’ web technologies will be more resilient, more stable, and better maintainable. Especially in the log run.

This series will be the blogs I write in documenting my process. I have no clue when to write something or when it is done, but we’ll see!

I have quite some experience with building React and Vue applications. Especially Vue is something that I love to work with. Because it combines a component based workflow and it feels really close to vanilla web technologies. Although in the end it is all compiled down to JavaScript, as a developer you write as if it were HTML, CSS and JavaScript. And to top it off, the tooling around these frameworks are top notch. Making the developer experience really really good in larger scale or complex sites / apps.

Then, Web Components caught my attention again. I did notice it when they were first introduced in 2011. But I never thought much of it. Until I recently did an experiment with them again. See if you can convert a VueJS component into a Web Component to use it in React. Turns out, you can do that! My colleague and I blogged about it here. That got me all exited again, especially since I noticed that LitElement and LitHtml are a thing now.

The problem of server side rendering

Server side rendering is still an issue with Web Components. React, Vue and all those other frameworks have this covered. But for the shadowDom, there is not a standardised way of serialising that to send it over the wire.

There are some experiments / solutions going on. Especially the @skatejs/ssr package by @treshugart is really close.

I tried the @skatejs/ssr package, and I got the example sort of working. Not quite, though, and I am sure that is due to my lack of proper trying that out. I looked through the source code of the package and realised that it wasn’t all that complicated. I could quickly see the concepts of what was going on. That together with the talk Trey Shugart gave at the Polymer summit in 2017 I was inspired to roll my own implementation. Take a look at that talk, it is really good.

I think that rolling my own implementation is something that will teach me quite a bit about the bread and butter of Web Components. And as far as I currently am it is a fun exercise!

If you want to check out my progress thus far, you can find the repo here

Back to basics to go forward

2018-02-15T00:00:00Z

My journey to become a better front-end developer

For a long time now I try to keep up with all the latest developments in our
front-end community. Learning React, a bit of Vue, diving into ES6,7,… You
know, like the rest of us.

Learning about all these frameworks, build tools and the like, I get
the feeling of missing out on a more important part of our trade. Because the
ultimate goal we all strive for, is to deliver a superb experience to our users.
And we do that by shipping HTML, CSS, JavaScript and a bunch of images. There is
still much to learn about those fundamental parts of the web. And that is where
I will focus my attention on.

And although I know these parts reasonably well, there is always room for
improvement. I want to get a deep an thorough understanding of these parts,
before I learn something that is build on top of that foundation.

I am going to approach this project like so:

Phase 1: Lay the foundation

A deeper foundational understanding of JavaScript
Get pretty intimate with CSS
Become really close friends with HTML

Phase 2: Build the walls

Brick by brick, Progressive Enhancement as a second nature
Preparing for the storm with Progressive Web Apps
Move things around with Animations

Phase 3: Put on the roof

Connect all the wires with HTTP

These are by no means set in stone (this is what first came to mind) but for now
it is my guide on this journey

PeterGoes.nl Blog

Updates for 02-2023

Looking at user stats increase my emotional connection with them

Looking at statistics from our actual users

I care much more about stats from our actual users than nation wide stats

Ditching the car turned out to be a very good idea

A few updates for 01-2023

I got genuenly exited to find out about a secret society!

The Set-Cookie and Cookie headers got me confused, but I think I get it now

Set-Cookie and Cookie are two different things

Reading response headers

Lessons learned

Should a website provide configurable accessibility options to users?

Update about the used example in this post

You can’t (rightfully) detect if accessibility features are enabled

Questions to answer

Do users want this level of customisation?

How does a user distinguish between this menu and other bad actors?

How do I make sure that this does not become a tracking data point?

Do you have any answers?

Review incoming WebMentions before publishing with Github Actions

Accepting Webmentions in the first place

Using a Github Action to create a Pull Request with new mentions

Fetching and storing new Web Mentions

The Github Action

Run the Github Action when a new mention is received

Use a Netlify function to call the Github Action

With great power comes great responsibility

We have the power to influence other peoples lives

Accessibility is not a nice to have, it is your responsibility

Binding behaviour to HTML with Web Components as Progressive Enhancement

Componentised UI with a HTML templating engine

Benefits

Closing thoughts

Why I signed the Overlay Factsheet

What the factsheet is all about

Why I signed

Learn about Web Components CSS selectors by server side rendering

New CSS selectors in Web Components

A quick recap of serialising the HTML

Differentiate SSR states in a Web Component

The problem

Life cycle methods in Vue

Life cycle hooks in Web Components

Using what I already have

Closing thoughts

My stab at rendering Shadow Dom server side

What is the Shadow Dom and why is it important?

Why do I want to render a Web Components Shadow Dom server side?

Serialising the Shadow Dom

The problem

Breaking down the problem

1. Do things with the loaded HTML

Get the modified HTML

Write the modified HTML to a new file

Hydrate client side

Things I did not cover

Experimenting with rendering Web Components

About this serie of posts

The problem of server side rendering

Back to basics to go forward

My journey to become a better front-end developer

`Set-Cookie` and `Cookie` are two different things