fusectore.dev

Exploring Identities in the AT Protocol via Bluesky

2025-01-04T00:00:00+00:00

The AT Protocol is an open, decentralized network for building social applications.

Bluesky, a relatively new social network, is built on top of the AT Protocol. It recently piqued my interest as I came across this fascinating research paper Bluesky and the AT Protocol: Usable Decentralized Social Media.

In this post, we’ll dive into one of the key elements of the AT Protocol: identities. We will start with Bluesky user handles, explore additional information about them, and see what else we can discover.

DID Resolution

A handle in Bluesky is associated with a DID, which can be resolved using DNS or HTTPS.

decentralized identifier (DID)

A globally unique persistent identifier that does not require a centralized registration authority and is often generated and/or registered cryptographically. […] Many—but not all—DID methods make use of distributed ledger technology (DLT) or some other form of decentralized network.

From Decentralized Identifiers (DIDs) v1.0

The DID for @jay.bsky.team can be resolved via a DNS record. Example with dig:

> dig _atproto.jay.bsky.team TXT
[...]
_atproto.jay.bsky.team. 14400 IN TXT "did=did:plc:oky5czdrnfjpqslsw2a5iclo"

If you don’t have dig installed, try an online tool like nslookup.io.

If no DNS TXT entry exists, the handle may resolve to a DID through a well-known HTTPS endpoint. Let’s take the example of @jamesgunn.bsky.social:

> curl https://jamesgunn.bsky.social/.well-known/atproto-did
did:plc:lotavzt36yanhfy3j3gpysyj

If you don’t have curl installed, you can just open the link in your browser https://jamesgunn.bsky.social/.well-known/atproto-did.

PLC

Once you have a DID, examine the part after the first colon. This is known as the method. The above two examples use plc, which stands for Public Ledger of Credentials. These PLC DIDs can be looked up in the PLC directory at plc.directory.

The AT Protocol also supports the web method, though we focus on plc here.

We can use curl again to get more information about a DID. Let’s take @jay.bsky.team’s DID as example.

> curl https://plc.directory/did:plc:oky5czdrnfjpqslsw2a5iclo/log | jq
[
  {
    "sig": "KuN3A61golVNSDU71wZKLP9lVuXk6YekJAz1lwDzrPsNTEWHBBW_8zSyV6pDxV4KiYXuAXlS1Ik47XkjQZ94mA",
    "prev": null,
    "type": "create",
    "handle": "jay.bsky.social",
    "service": "https://bsky.social",
    "signingKey": "did:key:zQ3shP5TBe1sQfSttXty15FAEHV1DZgcxRZNxvEWnPfLFwLxJ",
    "recoveryKey": "did:key:zQ3shhCGUqDKjStzuDxPkTxN6ujddP4RkEKJJouJGRRkaLGbg"
  },
  {
    "sig": "TY3ot8yFF-0LL8ceoRwSGaNQj0F1aj-IApYXRGT21G1fnznKHUHT1QE7c1aTrYd9PQLVvXUGag6CZ9EEeIKqgA",
    "prev": "bafyreidswhiwi4ljkl4es4vwqhkas3spmmktortqbp6lkrb5v7qqdfr3mm",
    "type": "plc_operation",
    "services": {
      "atproto_pds": {
        "type": "AtprotoPersonalDataServer",
        "endpoint": "https://bsky.social"
      }
    },
    "alsoKnownAs": [
      "at://jay.bsky.social"
    ],
    "rotationKeys": [
      "did:key:zQ3shhCGUqDKjStzuDxPkTxN6ujddP4RkEKJJouJGRRkaLGbg",
      "did:key:zQ3shpKnbdPx3g3CmPf5cRVTPe1HtSwVn5ish3wSnDPQCbLJK"
    ],
    "verificationMethods": {
      "atproto": "did:key:zQ3shXjHeiBuRCKmM36cuYnm7YEMzhGnCmCyW92sRJ9pribSF"
    }
  },
  {
    "sig": "WmQmobHA6abtu8xIkBSrDkkoMNjOxP1Tl_1zl1DbIZlA9kjOyxjX-J1rzwWVy3stdzMowpBeBnedkAug84n_RQ",
    "prev": "bafyreicgb25yf5fro22oyhtkbgerzr4nume4sx757r525skxdzpeoeseha",
    "type": "plc_operation",
    "services": {
      "atproto_pds": {
        "type": "AtprotoPersonalDataServer",
        "endpoint": "https://bsky.social"
      }
    },
    "alsoKnownAs": [
      "at://jay.bsky.team"
    ],
    "rotationKeys": [
      "did:key:zQ3shhCGUqDKjStzuDxPkTxN6ujddP4RkEKJJouJGRRkaLGbg",
      "did:key:zQ3shpKnbdPx3g3CmPf5cRVTPe1HtSwVn5ish3wSnDPQCbLJK"
    ],
    "verificationMethods": {
      "atproto": "did:key:zQ3shXjHeiBuRCKmM36cuYnm7YEMzhGnCmCyW92sRJ9pribSF"
    }
  },
  {
    "sig": "dhstt4uPua8OVs8TVzHTqtCnNMsBDN7kjxIBmTIYBDRkknrtprBJ_AISXFoBZyqfoxq2altp-vlRAPEKSh5zeg",
    "prev": "bafyreihmf7dapx27fexc7jwj4cdpbxcmhnnwo3x5vrpvbr6lclzj7gpsmi",
    "type": "plc_operation",
    "services": {
      "atproto_pds": {
        "type": "AtprotoPersonalDataServer",
        "endpoint": "https://morel.us-east.host.bsky.network"
      }
    },
    "alsoKnownAs": [
      "at://jay.bsky.team"
    ],
    "rotationKeys": [
      "did:key:zQ3shhCGUqDKjStzuDxPkTxN6ujddP4RkEKJJouJGRRkaLGbg",
      "did:key:zQ3shpKnbdPx3g3CmPf5cRVTPe1HtSwVn5ish3wSnDPQCbLJK"
    ],
    "verificationMethods": {
      "atproto": "did:key:zQ3shtJpFGgEG3tv3ERKvjo7VHbjDPVyvjYvW7gpie49rtNtc"
    }
  }
]

Again, feel free to click on it instead, https://plc.directory/did:plc:oky5czdrnfjpqslsw2a5iclo/log.

This returns a list of all operations that were performed on this DID. The first entry is the genesis operation — the one that created the DID. It is the only one where prev (previous entry) is null, because it is the first one.

Note that the first entry of this DID’s log has a different format. This entry uses the legacy format. Currently, newly created DIDs use the new format (subsequent records), but @jay.bsky.team has been around for a while (the account belongs to Bluesky’s CEO).

If you look through the logs you will notice a couple of changes that were made to her identity over time:

the handle was changed from jay.bsky.social to jay.bsky.team (note that the legacy format used handle whereas the new format uses alsoKnownAs)
- the handle now includes at://
the personal data server changed from https://bsky.social to https://morel.us-east.host.bsky.network

Signatures

Each entry in the log includes a sig field, which contains the signature. This cryptographic signature is derived from the other fields in the entry. For the genesis operation, the rotationKeys (or signingKey for legacy) field contains the public key(s) of the key pair(s) with which the entry was signed.

Subsequent operations need to be signed with the key from the previous entry. Additionally, the prev field references the previous entry.

Creating a DID

The identifier in the DID, which is the last part of the colon-separated string, is constructed from the genesis operation. For @jay.bsky.team:

the identifier is oky5czdrnfjpqslsw2a5iclo
the genesis operation is this JSON document:

{
  "sig": "KuN3A61golVNSDU71wZKLP9lVuXk6YekJAz1lwDzrPsNTEWHBBW_8zSyV6pDxV4KiYXuAXlS1Ik47XkjQZ94mA",
  "prev": null,
  "type": "create",
  "handle": "jay.bsky.social",
  "service": "https://bsky.social",
  "signingKey": "did:key:zQ3shP5TBe1sQfSttXty15FAEHV1DZgcxRZNxvEWnPfLFwLxJ",
  "recoveryKey": "did:key:zQ3shhCGUqDKjStzuDxPkTxN6ujddP4RkEKJJouJGRRkaLGbg"
}

Steps:

encode the genesis operation in DAG-CBOR (this is a concise binary format and looks a bit like JSON)
create a SHA256 hash of the encoded document
encode the hash with base 32 and make its letters lowercase.
take the first 24 characters
add did:plc: as a prefix to create the 32-character DID

Verification

Based on what we’ve learned so far we can verify a handle from Bluesky. For example, if we want to check @jay.bsky.team:

resolve DID via DNS: did:plc:oky5czdrnfjpqslsw2a5iclo
look up DID in PLC directory
the handle in the alsoKnownAs (or handle for legacy) field must match jay.bsky.team
verify all operations using the public keys and signatures
reconstruct the DID from the genesis operation and ensure it is the same as the one we found in the DNS TXT record

Can we trust the PLC directory though? Say a malicious actor has taken over control and modified all entries in the log. The hacker even constructed the entries in the log so that all the signatures appear valid. How would we still know the PLC directory is compromised? Remember that the DID can be reconstructed from the genesis operation. If that were modified, we’d end up with a different DID from the one we found in the DNS TXT record.

Summary

This blog explores the AT Protocol, the decentralized foundation of social applications like Bluesky, with a focus on user identities and their technical underpinnings. It explains how Bluesky user handles are associated with Decentralized Identifiers (DIDs), which can be resolved using DNS records or HTTPS endpoints. Detailed examples demonstrate how tools like dig and curl retrieve DID data, revealing the method (plc) and structure used to catalog them in the Public Ledger of Credentials (PLC). The post also examines the DID lifecycle, showcasing how historical operations and changes—such as updates to handles or servers—are logged and accessible for transparency.

There are several aspects that we did not investigate, such as:

the web method for DIDs
tombstone operations in the log
recovery
how exactly to construct the prev entry in the DID document
what the other fields in the DID document mean

If you are curious, the links in the next section will help you answer these questions.

Links & Further Reading

How well do you know GitHub Actions?

2022-09-25T00:00:00+00:00

During one of my assignments, I worked with GitHub Actions pretty much every day. I implemented workflows, created new actions, and helped people migrate their projects from jenkins to actions.

As much as I like actions – and I like them a lot – there are some things that caught me off guard. I have collected some of these things, both for other people to let them know, and for myself as a future reference.

How well do you know actions? Can you answer all of these questions correctly?

ℹ️ Assume that all YAML and code is valid/compiles

Output from a previous job

What does the following print?

jobs:
  jobOne:
    runs-on: ubuntu-latest
    outputs:
      foo:  ${{ steps.foo.outputs.foo }} 
    steps:
      - run: echo '::set-output name=foo::bar'
        id: foo
      
  jobTwo:
    needs: jobOne
    runs-on: ubuntu-latest
    steps:
      - run: echo hello
        
  jobThree:
    needs: jobTwo
    runs-on: ubuntu-latest
    steps:
      - run: echo  ${{ needs.jobOne.outputs.foo }} 

Solution: It prints an empty line, because if you want to access an output from a job, you need to list it as a dependency. Outputs from transitive dependencies cannot be accessed.

To fix this, make the third job depend on the first job explicitly needs: [jobOne, jobTwo]

Exec explodes

GitHub provides action developers with a toolkit to develop actions in JavaScript. One of the functions in there is called exec and can be used to run a command. Here’s its signature:

/**
 * Exec a command.
 * Output will be streamed to the live console.
 * Returns promise with return code
 *
 * @param     commandLine        command to execute (can include additional args). Must be correctly escaped.
 * @param     args               optional arguments for tool. Escaping is handled by the lib.
 * @param     options            optional exec options.  See ExecOptions
 * @returns   Promise<number>    exit code
 */
export async function exec(
  commandLine: string,
  args?: string[],
  options?: ExecOptions
): Promise<number>

In an action, you might use it like so:

const returnCode = await exec.exec('node', ['index.js']);
if (returnCode != 0) {
    console.error('Uh! Something did not quite work :(')
}

Do you see any problems with that code?

The problem is that exec does not return a non-zero return code if the command fails. Instead, it returns a rejected promise.

So if you want to handle the error case, you need to wrap it in a try-catch block.

While this behavior can be changed by passing ignoreReturnCode as the third argument ExecOptions, the default behavior is very surprising. I have seen many examples of people checking the return code without knowing how this function really behaves.

Nevertheless, I still think throwing an error by default is probably not a bad choice, given that (A) many people ignore error handling and (B) you probably want to abort on error.

Push for all?

Say you’re working on your open source project, and you just started using actions to make sure the build always passes:

on: push

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      # etc..

The push trigger seems to do the trick in all cases: You can push to a branch (or pull request) and the build runs. You can push to the main branch and the build runs.

But there’s a scenario where it won’t run. Do you know when?

Solution: If I fork your project and create a pull request, then push won’t trigger. If you want the PRs of external contributors to be validated with your workflows as well, you need to use the trigger pull_request. And once you do that, you also need to make sure the push only triggers on main, because otherwise your own pull requests will trigger the workflow twice (both from push and pull_request):

on:
  pull_request:
  push:
    branches: [main]

When talking about running actions from contributors, it is important to mention that:

You can control how actions are triggered in pull requests from forks by Approving workflow runs from public forks
Secrets are generally not available in PRs from forks, except for $GITHUB_TOKEN, which is read-only. There’s a trigger called pull_request_target to work around that, but you need to be careful as this open up the gates for potential vulnerabilities.

Too much information, GitHub

Say I have an open pull request based on a branch called my-feature, and the following workflow is triggered. What does it print?

on: pull_request

jobs:
  branch:
    runs-on: ubuntu-latest
    steps:
      - run: echo  ${{ github.ref_name }} 

I’ll even give you the docs on github.ref_name:

The short ref name of the branch or tag that triggered the workflow run. This value matches the branch or tag name shown on GitHub. For example, feature-branch-1.

I’m going to guess you said my-feature, but unfortunately you’re wrong.

It is going to print the PR number followed by /merge. For example, it would print 18/merge for the 18th pull request in your repository.

Why is that? 🤔

The simple explanation is that when you create a pull request on GitHub, they create some additional internal(!) branches to make it easier for them to manage your pull request and tell you how it compares to the branch you want to merge into. As a result of that, when you run a workflow on a pull_request event, it runs on one of these “internal” branches rather than the one you just pushed. In terms of contents on your branch, this doesn’t make a difference – you’re still running your builds and tests on your contents – it’s just the branch name that is different.

Unfortunately, this implementation detail is exposed here.

Please see this answer on stackoverflow for more information.

Note that $GITHUB_SHA is not what you’d expect either

So how can you access the name of your branch in a pull request? It’s in the payload from the event:

on: pull_request

jobs:
  branch:
    runs-on: ubuntu-latest
    steps:
      - run: echo  ${{ github.event.pull_request.head.ref }} 

Bonus: Am I there or not?

Consider this reusable workflow:

on:
  workflow_call:
    inputs:
      my-input:
        description: optional input
        required: false
        type: string

jobs:
  greet:
    runs-on: ubuntu-latest
    steps:
      - uses: my/action@v1
        with:
          my-input:  ${{ inputs.my-input }}  

Let’s say the action my/action@v1 has an optional input my-input which has a default value my-value. Now if you were to call the reusable workflow as shown in the following example, what value would be passed to my/action@v1? Would the default be used?

jobs:
  jobA:
    uses: org/repo/.github/workflows/workflow.yml@v1

Solution: An empty string is passed to my/action@v1 as my-input.

The way this works is that if no value is passed to the reusable workflow, then the input my-input is still passed to the action, but it’s passed as an empty string. And even though we usually see an empty string as “no value passed”, the default value is not used here, because we did pass something.

If you wanted to use the default value if no input was passed, then you’d have to invoke the action differently based on whether the input was passed to the reusable workflow.

steps:
  - uses: my/action@v1
    if: inputs.my-input
    with:
        my-input:  ${{ inputs.my-input }} 
  - uses: my/action@v1
    if: inputs.my-input == ''

This has been a long-standing issue in the runner, but there’s likely no easy solution that does not break existing workflows.

Notes from the Enlightened - Productivity Tip with Todoist

2022-06-26T00:00:00+00:00

Earlier this year, after three years, ten months, two weeks, and three days after having created my Todoist account, I reached the Enlightened status, which is the highest level you can reach in Todoist’s gamification system. According to Todoist, only 0.05% of their users reach this status.

I have been described as productive or structured by my peers. While I don’t think I’m as “productive” or “structured” as some might think, I still believe todoist helps a lot with organizing everyday tasks.

In this post, I’m going to describe how I use Todoist to organize my work and life.

Writing Things Down Frees My Mind

A few years ago, I read this book called Getting Things Done. I don’t remember much, but one thing stuck with me: Imagine you need to mail an important letter tomorrow. This might prevent you from falling asleep easily, because you keep thinking about what would happen if you forgot to mail it.

In these situations, I always add a task to Todoist and set a reminder for when I need to do it. This helps me free my mind and focus on something more important – or nothing at all, when I’d like to fall asleep.

Stick With One Tool

Are you reading Product Hunt? Hacker News? There’s a new tool for keeping tasks almost every day.

While trying them might be very tempting, I am convinced this is actually counter-productive. In fact, this is probably a form of procrastination: Instead of using one imperfect tool, you keep switching between tools only to discover that they’re not perfect either.

Todoist is pretty simple. At least on the surface, it’s really just a piece of software that lets you create a task and set a due date. I don’t think more is needed to organize my life: A list of tasks, due dates, and potentially some form of categorization.

Groom Your Inbox

Todoist comes with several views. The one I use most often is “Today”. In there are the tasks I want to finish today. Another one is the “Inbox”. Those are the tasks without a date.

In terms of tasks that land in the Inbox, because they miss a due date, I have decided for myself that that’s a red flag. There is no task that has no due date. There might be a task that is not important, but even that task probably needs to be done by a certain date.

Therefore, about once per month, I scroll through my Inbox and make sure every task has a due date. Sometimes, I would see a task where I actually forgot to set a date, and I’d fix that. However, what happens more often is that there are things in my Inbox, which are not really tasks, but rather just some notes I jotted down when I was on the go. I move those to the respective project, but more on that in the next section.

Gather Notes In Projects

I don’t just use Todoist for things I need to do. I also use it to collect articles I’d like to read, recipes I’d like to try, things I’d like to do, etc.

For these types of notes, I create a project. For example, I have a project with restaurants I have been to. For each restaurant, I add a short comment with my impression and where it is located.

Procrastination on HackerNews

HackerNews is my favorite place to slack away. If I’m working on a task I don’t like, suddenly every article on HackerNews seems very interesting to read.

Instead of actually reading them, I just add them to a project in Todoist called “Articles to Read”. Whenever I do have some time to read (e.g. traveling on a train), I will look at this list of articles and start reading. In this situation, I often find the headlines much less appealing, and I just remove the task without reading the article.

Add People Tags

There are always things I need to discuss with other people.

For example, while working on accounting for my company, I might have a question for my accountant. Since it is usually not possible to bring these questions up right away, I usually create a task in Todoist and use a tag with the person’s name I want to discuss it with. This allows me to pull up a list of questions for my accountant when I am on the phone with her.

Or, while working on a project at work, I may come up with an idea to improve the way we work. I’m sure we all had. But then when I have this meeting with my boss and the “uhm, so what else?” - question comes up, I now actually have a list of points to discuss. Instead of “uhm, yeah, I had something, but now I can’t remember”

Set Up Recurring Tasks

Remember the grooming I mentioned above? How do I not forget to do that? There’s a task that repeats every month.

Another example of a recurring task I have is on Monday morning to look at my week’s schedule.

I actually have a lot of recurring tasks. Sometimes, I would even create them as a temporary reminder of things I need to do for a short period of time. For example, I would really like to stretch every day. In order to get into the habit of doing that, I have created a task that repeats every day at 1 p.m.

Use Magic Text To Schedule

As I’m working on a Mac these days, I create most of my tasks with the keyboard shortcut CMD (⌘) + CTRL + A. When adding a task, I also immediately set a due date by just typing it in and Todoist automagically recognizes it.

If you type a # into the field, a dropdown shows up with all projects. I use this to quickly categorize a task.

So imagine someone sends me an interesting link via Slack and I don’t want to read it right away:

Open the “Quick Add” on my Mac with the shortcut described above
Paste the link (Todoist automatically pulls in the title of the page)
Type a # and find the respective project by typing the first few letters of “Articles”
Enter 🤓

Make Packing Lists Using Templates

I used to return to my home country on a regular basis when I was living abroad. Here’s my problem: I’m always terrified that I’m going to forget something (and it often happened). So what I did with Todoist was create a project with all the things I needed to pack (one task per item).

Now, whenever I am about to travel, I copy the project and start ticking off tasks as I pack my backpack. This makes the original project kind of a template, and the copy is the one I actually work on.

If I do forget something, I just need to make sure to add it to the template project, and next time I won’t forget.

Make Mundane Tasks More Fun

Sometimes I need to work on a mundane task, and I have difficulty focusing. When I become aware of that, I sometimes create a few tasks or subtasks in Todoist. By splitting off one big, not-so-interesting task into several still-not-very-interesting subtasks, I at least get the feeling of accomplishing something whenever I can tick off one of the smaller tasks.

This way, I gamify my work, and it helps me get even the boring stuff done.

Conclusion

By now, you probably guessed that Todoist is pretty central to my life. And there would be more to talk about, but I don’t want to make this article too long.

I hope this list inspires you to use some sort of todo app as well and helps you stay on top of your tasks.

If you have any suggestions or tricks for Todoist or productivity in general, please get in touch with me via email.

RSA/ECB/PKCS1Padding in Node.js

2021-12-17T00:00:00+00:00

In Java, you can encrypt a string using the cryptic RSA/ECB/PKCS1Padding algorithm. If you try to do the same thing in Node.js, you’ll soon realize that this is not possible. Not because the equivalent does not exist in Node.js, but because the “algorithm” RSA/ECB/PKCS1Padding does not really exist and the identifier in Java is misleading.

What is the problem?

Let’s take a look at the following code in Java:

public byte[] encrypt(String message, PublicKey publicKey) throws Exception {
    Cipher cipher = Cipher.getInstance("RSA/ECB/PKCS1Padding");
    cipher.init(Cipher.ENCRYPT_MODE, publicKey);
    return cipher.doFinal(message.getBytes());
}

This method takes a plaintext message and encrypts it using the specified public key. Looking at the invocation getInstance on Cipher we get the idea that the algorithm used is RSA/ECB/PKCS1Padding. But what is this?

“RSA” sounds like it is using asymmetric encryption, but “ECB” is a block cipher mode of operation typically used to encrypt multiple blocks of data using a symmetric key.

So what does this cipher actually do? Does it create a random key for the symmetric encryption and then encrypt that key with the public key appending it to the message as is done in hybrid encryption?

Well, no. The name is just misleading. In fact, looking at the source code of the class Cipher, we see that the actual implementation for the encryption is in a class called RSACipher.java. And inside that class, the following method reveals it all:

// modes do not make sense for RSA, but allow ECB
// see JCE spec
protected void engineSetMode(String mode) throws NoSuchAlgorithmException {
    if (mode.equalsIgnoreCase("ECB") == false) {
        throw new NoSuchAlgorithmException("Unsupported mode " + mode);
    }
}

So ECB is used as an identifier, but has no actual meaning or influence. Passing anything else makes this method fail.

What does really happen?

We haven’t talked about padding yet. I won’t go into too much detail, but what you need to know is that encryption works on fixed-size blocks of data. For example, if you have a message of 30 bytes and the algorithm encrypts 45 bytes at a time, then you need to add 15 bytes to the message. When decrypting, that algorithm also needs to know that these 15 bytes were only added as padding and are not part of the original message.

What if the message is bigger that the amount of bytes that are encrypted at a time? Usually, you’d encrypt the message block-by-block combing the individual blocks together. This is where ECB would come into play. As we saw before, ECB is ignored though. Therefore, this method would throw an exception when called with a message that is bigger than one block.

Knowing that ECB is ignored, what this algorithm really does is encrypt the message with the public key using PKCS1 padding.

How can the equivalent be done with Node.js?

Let’s say we want to do the same in Node.js. We wouldn’t be the first one to search online for “how to do RSA/ECB/PKCS1Padding in Node.js”. This has been asked in a popular node library’s issue tracker but also on StackOverflow for python

All answers point to the same: It does not make sense and therefore does not exist. However, knowing what we learned above, what we actually need is a way to encrypt a message with RSA using PKCS1 padding. This makes the search easier.

In Node.js, we can use the module node-forge. You can install it using the following command:

> node install node-forge

And then we can use it to encrypt the message:

function encrypt(message, publicKey) {
    var bytes = forge.util.createBuffer(message, 'utf8').getBytes();
    return publicKey.encrypt(bytes, 'RSAES-PKCS1-V1_5');
}

This is assuming that the specified publicKey has been created with forge as well. There are several examples in their documentation how a public key can be created.

Conclusion

The naming for this algorithm is really misleading. This makes the search for the equivalent implementation much more difficult than it should be. Instead of finding what we’re looking for, we had to take a look at the source code to see what really happens.

Once it’s clear what’s really happening, the solution turns out to be quite simple though.

Fancy GitHub Actions with Problem Matchers

2021-11-19T00:00:00+00:00

GitHub Actions is the CI/CD platform for GitHub. It’s a powerful tool that makes it easy to run CI/CD pipelines on GitHub.

For (aspiring) authors of Actions, there is a little known tool that can help you propel your Action from one that shows some output in the logs to one that directly annotates problems in your code making it visible in pull requests.

I’m talking about Problem Matchers and the annotations they can create:

What is a Problem Matcher?

A Problem Matcher is a small piece of configuration that tells GitHub Actions what lines of your logs correspond to what lines of your code.

Imagine a tool that creates error messages like the following:

> json-validator myconfig.json
WARN: myconfig.json:1:9: Missing attribute 'age'
ERROR: myconfig.json:1:4: Unexpected token '}'

These logs contain all information necessary in order to create annotations as shown above:

each line starts with the severity of the message
it then specifies the filename and location where the problem occurred
and the last part of the line is the actual problem

A Problem Matcher tells GitHub exactly this: how bad is the problem (severity), where did it occur (filename and line number), and what is the problem itself (message).

How can I create a Problem Matcher?

Let’s say you want to create a Problem Matcher for the hypothetical json-validator above. There are two steps you need to follow in order to create and use such a Problem Matcher.

Step 1: Define the Problem Matcher

A Problem Matcher is defined in a JSON file and consists of an owner and a pattern that defines how to scan the logs and extract the relevant information.

The following is an example Problem Matcher for the json-validator.

{
  "problemMatcher": [
    {
      "owner": "json-validator",
      "pattern": [
        {
          "regexp": "^(ERROR|WARN|INFO):\\s(.+):(\\d+):(\\d+):\\s(.+)$",
          "severity": 1,
          "file": 2,
          "line": 3,
          "column": 4,
          "message": 5
        }
      ]
    }
  ]
}

I have given it a name, json-validtor and defined the regular expression in the regexp field. This regex says:

log lines start with ERROR, WARN or INFO
after a colon and a space , there is a filename
after another colon, there is a line number
after another colon, there is a column number
after another colon, there is a message

With the attributes below the regexp attribute, we’re defining which match group (these are the patterns in brackets) of the regular expression map to which attribute. For example, by defining severity: 1, we’re telling the regex that the first match group is the severity.

And that’s it! All we now need to do is install the Problem Matcher.

Step 2: Install the Problem Matcher

With the JSON file from the first step stored as matcher.json, we can go ahead and install the matcher either in a workflow or as part of an Action. This can be done with the command add-matcher, like so:

steps:
  - run: echo "::add-matcher::matcher.json"
  - run: json-validator myconfig.json
  - run: echo "::remove-matcher owner=json-validator::"

With these steps, we’re first installing the Problem Matcher, then running the Action, and eventually removing the Problem Matcher again.

Note that a Problem Matcher is removed by specifying the owner of the matcher. Removing it makes sure the Problem Matcher is not applied to unrelated Actions that emit similar log files.

Conclusion

Hopefully this article gives you a good overview of how Problem Matchers work and how to use them.

If you want to see them used in practice, checkout out the setup-node action.

If you’re looking for something to play around with, or if you want a head start for a simple action that uses Problem Matchers, you might want to take a look at my bash-action.

Troubleshooting Netctl under Arch Linux

2016-12-28T00:00:00+00:00

I recently had an issue connecting to a WLAN with my ArchLinux/Systemd/Netctl notebook. While analyzing it, I found a few ways to troubleshoot issues with netctl.

For the following instructions and examples, please note that my wlan interface is called wlp2s0 and may differ on your system. You may get a list of interfaces with the command

$> ip link

Is your profile loaded?

After I copied and adapted one of the profiles from /etc/netctl/examples and tried to switch to it, I got the following error message:

Profile 'wlp2s0-33C3' does not exist or is not available

The problem was that I did not reload netctl-auto. This is how that can be done:

$> sudo systemctl restart [email protected]

It is usually handy to have the journal log running in a separate console while looking into issues of this sort. When reloading netctl-auto for example, you could have the journal log running with

$> sudo journalctl -xef

If the profile was successfully loaded, you should see the output

Dec 28 13:24:28 asus-rethab netctl-auto[3204]: Included profile 'wlp2s0-33C3'

Issues with the profile?

The ESSID of the network I was trying to connect to was 33C3. For some reason, I forgot to put this into quotes in the netctl profile so it would simply not be activated and keep falling back to some other default profile. I figured this out by instructing netctl to tell me more about what it is doing with the environment variable NETCTL_DEBUG like so:

$> NETCTL_DEBUG=true sudo netctl-auto switch-to wlp2s0-33C3

This way I learned that wpa_cli was trying different networks and eventually seemed to fallback to the default. So I ran wpa_cli manually to see which options it tried:

$> sudo wpa_cli -i wlp2s0 list_networks

This listed me all familiar networks plus one that was named 3\xc3 which lead me to suspect that something with the ESSID must be wrong.

Practical Dependency Injection for the Play Framework using Guice: Part 1

2016-11-26T00:00:00+00:00

This is the first part in a series of articles on dependency injection for the play framework using guice. As soon as I’ve written the next part, it will be linked here.

Please note that while the examples are written in Scala, the principles apply for Java likewise.

Furthermore, please be aware that Guice is not the only framework to do dependency injection with. It is probably the most prominent one in the play ecosystem though. Guice does so called runtime dependency injection. This means dependencies are resolved at runtime. An alternative approach is compile time dependency injection, which essentially means if a dependency cannot be resolved, the code doesn’t even compile. Please read the play documentation for more on the latter: Compile Time Dependency Injection

Why Dependency Injection?

At the time of writing, version 2.5 is released and the play framework is in the process of moving away from global state (What exactly is that global state?). While older version heavily relied on the function Play.current, this has now been deprecated and people are looking for ways to restructure their applications to no longer depend on this.

Accessing the currently running application used to be the central access point for a variety of components such as Configuration, ActorSystem, Connection, WSClient and many more. Therefore, we now need a new way to access them.

A Simple Example

The following example shows how we could use the configuration in a controller where we previously might have used Play.configuration. Adding the annotation javax.inject.Inject directly after the class name allows us to specify a number of components that we would like an instance to be injected.

class MyController @Inject() (config: Configuration) {
  val maxSize = config.getInt("maxSize")
}

What just happened?

If we look at the above example again, we can see that our controller is a simple class that takes one parameter. But how does the framework know which exact parameter is to be passed in there? This is defined by using so called bindings. A binding can be roughly formulated by saying if a thing of type A needs to be injected into a constructor, which concrete instance of A shall be taken.

A number of bindings are provided by the play framework. For example, as seen above, the play framework defines (in a class called BuiltinModule.scala, for the impatient) which concrete instance of Configuration is to be injected if a user requests a Configuration. An (incomplete) list of the components that are provided by the play framework can be found in the 2.4 Migration Guide.

Migrating Companion Objects

A consequence of injecting instances is that code that requires such a component can no longer live in a companion object as it could before. Let’s say we have a service that has to access the configuration and a controller that uses the service:

object MyService {
  def getMaxSize: Option[Int] = Play.configuration.getInt("maxSize")
}

class MyController {
  val maxSize = MyService.getMaxSize 
}

Now that we need to avoid static access to the configuration and inject it instead, we also need to migrate the service to a class. But then how does the controller access the service? Just like the service injects the configuration, the controller now needs to inject the service:

class MyService @Inject() (config: Configuration) {
  def getMaxSize: Option[Int] = config.getInt("maxSize")
}

class MyController @Inject() (myService: MyService) {
  val maxSize = myService.getMaxSize
}

Why go through all this Hassle?

As the above example with the service and the controller has shown, using dependency injection increased the size of the code and made it a bit more complicated as when compared with directly invoking the method on the companion object.

One big win of dependency injection, however, is testability. Just imagine for a second how you’d write a test for the method getMaxSize in the companion object. Since it accesses the configuration from the currently running application, you’d need to make sure that an application is running when you’re testing that method. Think about this: You need an entire play application running just to test a simple method!

Injecting the dependencies greatly simplifies that. You can now instantiate the service directly in your test and pass in the configuration required for your test - scenario. You could even easily test the behavior if the key maxSize does not exist (the example uses specs2).

class MyServiceTest extends Specification {
  "getMaxSize" should {
    "return the size" >> {
       val config = Configuration("maxSize" -> 4)
       val service = new MyService(config)
       service.getMaxSize must_=== Some(4)
    }

    "return none if not configured" >> {
       val config = Configuration()
       val service = new MyService(config)
       service.getMaxSize must beNone
    }
  }
}

Conclusion

In this first article, I have tried to shed some light into why the play framework uses dependency injection in the first place as well as a few examples on how to use dependency injection within an application and the benefits that come along with it.

Stay tuned for the following parts where we’re going to look at more realistic examples, learn how to define bindings and much more!

Links and Further Reading

Google Guice 101, Bob Lee Jesse Wilson: https://www.youtube.com/watch?v=FFXhXZnmEQM
Guice Docs: https://github.com/google/guice/wiki/Motivation
Assisted Inject: https://github.com/google/guice/wiki/AssistedInject