A couple of weeks ago I read the fantastic post Parse, don’t validate by Alexis King. It is a longer post and worth reading, but I want to summarize what I got out of it. For me, the core ideas were:

1. It is better to parse input into valid values rather than validate

2. People will forget to use your validation.

3. After parsing, the rest of the code must be able to rely on it being correct.

These ideas came back to my mind recently while learning Go for a new project at work.

An example in Go

I decided to pick up Learn Go with Pocket-Sized Projects1 since I like a “closed source” as the starting point when learning something new. While reading chapter 6, I remembered reading King’s article and thought this was exactly the situation she described. In that chapter, the project is a currency converter. It contains the type Amount.

type Amount struct {
	quantity Decimal
	currency Currency
}

The details of the type are not important, it is used to save monetary amounts in minor units of the currency (to avoid rounding errors with float). The authors notice a thing: you can create invalid instances of Amount:

[…] it seems wise to check that we’ll get the expected output […] Following are the limitations:

• The number can’t be higher than 1012, or it will lose precision due to the floats.

• The number’s precision must be compatible with the currency’s precision.

The chapter goes on by creating a validate() method on Amount:

func (a Amount) validate() error

There is nothing inherently wrong with that, but there is a flaw: you can create invalid Amount in first place.

So in the sense of Parse, don’t validate, it would be wiser to introduce a Parse() method as the only way to create Amount:

func Parse(quantity Decimal, currency Currency) (Amount, error) {
	switch {
	case quantity.subunits > maxDecimal:
		return Amount{}, ErrTooLarge
	case quantity.precision > currency.precision:

		return Amount{}, ErrTooPrecise
	}

	return Amount{quantity: quantity, currency: currency}, nil
}

The code itself is simple, but it shows how moving checks to the point of construction can remove an entire class of problems from the rest of the program.

Closing

This post only scratches the surface, and Alexis King does the heavy lifting in the original article. I highly recommend reading Parse, don’t validate in full.

I hope you got something useful out of this field note. If you have thoughts, questions, or similar examples, feel free to reply or reach out!

Leave a comment

✍️ About these field notes
This newsletter is my notebook for things I learn and find useful. Not everything needs a long post, so from time to time I will publish short “field notes” like this one instead of letting them sit unread in my notes app.

Engineers read more code than they write, but they may read even more log messages. We read them to see if everything works and, if not, to locate the problem. More often than not, it is hard to find a needle in a haystack, especially when things work in 99% of cases but fail for a single user.

To make the haystack smaller, it is vital to filter log messages efficiently, for example by adding the userID. Doing this by hand for every log message would be tedious, but we can leverage the mapped diagnostic context (MDC).

In this post, I will show how to automatically add useful information to the MDC based on path and query parameters in Ktor. With just a few lines of code, you can enrich log messages with valuable context. But first, let us take a look at what MDC actually is.

TL;DR: If you already know what MDC is and just want the Ktor plugin, feel free to skip ahead to the section on implementing the PathParameterMDCPlugin.

You can find the complete code on GitHub.

Understanding Mapped Diagnostic Context

Before we start, I want to make it clear that the following is based on SLF4J / Logback. MDC is not a concept exclusive to Logback, but this is the setup I have experience with.

So, what is MDC? Simplified, it is a thread-local Map<String, String>. The contents of this map can be added to each log message without explicitly doing so every time; this is configured once in the logger configuration.

Here is a simple example. First, we add %X to the Logback pattern to output the MDC content:

<configuration>
    <appender name=”STDOUT” class=”ch.qos.logback.core.ConsoleAppender”>
        <encoder>
            <pattern>MDC: [ %X ] %-5level - %msg%n</pattern>

        </encoder>
    </appender>
...
</configuration>

We can use MDC’s methods:

import org.slf4j.LoggerFactory
import org.slf4j.MDC

fun main() {
    val logger = LoggerFactory.getLogger(”logger”)
    logger.info(”Hello World!”)
    MDC.put(”key”, “value”)
    logger.info(”Hello again!”)
    MDC.put(”another key”, “new value”)
    logger.info(”Goodbye!”)
}

Which gives us:

MDC: [  ] INFO  - Hello World!
MDC: [ key=value ] INFO  - Hello again!
MDC: [ another key=new value, key=value ] INFO  - Goodbye!

It is easy to see how this can be useful. If we can automatically add the userID to every log message, filtering logs becomes trivial.

However, especially with Kotlin and coroutine-heavy Ktor, there is a problem: the MDC is bound to the thread. We can see this in the following example:

MDC.put(”userID”, “abc”)
logger.info(”Start order creation”)
coroutineScope {
    launch {
        MDC.put(”orderID”, “123”)
        logger.info(”Created order”)
    }

}
logger.info(”Publishing order”)

The output looks like this:

MDC: [ userID=abc ] INFO  - Start order creation
MDC: [ orderID=123 ] INFO  - Created order
MDC: [ userID=abc ] INFO  - Publishing order

We lose important information about how orders relate to users if we rely on the static methods. Fortunately, we can fix this by leveraging the correct coroutine context. To do so, we need to add a dependency and fill the MDC differently.

First, add the following dependency to your build.gradle.kts:

 implementation("org.jetbrains.kotlinx:kotlinx-coroutines-slf4j:1.10.2")

Then adjust the code from above:

...
import kotlinx.coroutines.slf4j.MDCContext
import kotlinx.coroutines.withContext

suspend fun main() {
    // ...
    coroutineScope {
        launch(MDCContext()) {
            MDC.put("orderID", "123")
            logger.info("Created order")

        }.invokeOnCompletion { logger.info("Publishing order") }
    }
    logger.info("End order creation") // new here
    }
}

This now produces more useful logging information:

MDC: [ userID=abc ] INFO  - Start order creation
MDC: [ userID=abc, orderId=123 ] INFO  - Created order
MDC: [ userID=abc, orderId=123 ] INFO  - Publishing order
MDC: [ userID=abc ] INFO  - End order creation

We can make two observations:

• userID is now propagated to the coroutine, as intended.

• Additions to the MDC are not propagated outside of the coroutine implicitly. There is a solution for this, which you can find in the KDoc.

Armed with that knowledge, let us take a look at how to fill the MDC based on your routes.

Deriving MDC Values from Routes

Our goal is to add any useful information present in the routes to the MDC.

For example, your application might have the following routes:

get("/orders/{orderID}/items/{itemID}") {
    logger.info("Processing order")
    call.respond("")
}

get("/items/{itemID}") {
    logger.info("Processing request for item")
    call.respond("")
}

For each call to /orders, we want both orderID and itemID in the MDC; for /items, only itemID. There are several ways to achieve this. The simplest approach is to add these values to the MDC manually, as shown above. This works fine for two or three routes but quickly becomes a hassle as the number of routes grows.

Another option is to leverage the CallLogging plugin, which we will look at next.

Using CallLogging to Populate the MDC

If your routes are well defined, you can configure CallLogging to populate the MDC. Well defined means that the path segment before a given ID is always the same. For example, items/ always precedes the {itemID} path parameter. This is necessary because CallLogging does not know about path parameters. Instead, you need to parse the resolved URL and extract the parameters yourself.

fun extractPathParameter(url: String, paramName: String) = when (paramName) {
    “itemID” -> “/items/([^/]+)”.toRegex().find(url)?.groupValues?.get(1)
    “orderID” -> “/orders/([^/]+)”.toRegex().find(url)?.groupValues?.get(1)
    else -> null
}

You can then use this function in the CallLogging plugin:

install(CallLogging) {
    mdc(”itemID”) { call ->
        extractPathParameter(call.request.uri, “itemID”)
    }
    mdc(”orderID”) { call ->
        extractPathParameter(call.request.uri, “orderID”)
    }
}

Note that call.request.uri contains the already resolved URL, which is why the extractPathParameter function is needed. With this setup, calling the endpoints yields:

MDC: [ itemID=123 ] INFO  - Processing request for item
MDC: [ itemID=123 ] INFO  - 200 OK: GET - /items/123 in 240ms
MDC: [ itemID=789, orderID=456 ] INFO  - Processing order
MDC: [ itemID=789, orderID=456 ] INFO  - 200 OK: GET - /orders/456/items/789 in 2ms

Nice, we get our IDs into the MDC without changing the routes. However, this approach only works if your routes are predictable and parameters can be reliably extracted from the resolved URL. In older applications with many past maintainers, the API schema is often less consistent. You could try to handle all cases in extractPathParameter, but that quickly becomes tedious. Luckily, there is another solution.

Implementing a Custom MDC Plugin

So assume the orders endpoint looks like this, using item instead of items:

get(”/orders/{orderID}/item/{itemID}”) {
    logger.info(”Processing order 2”)
    call.respond(”“)
}

Now, adding itemID to the MDC will still work for /items, but not for /orders:

MDC: [ itemID=123 ] INFO  - Processing request for item
MDC: [ itemID=123 ] INFO  - 200 OK: GET - /items/123 in 483ms
MDC: [ orderID=456 ] INFO  - Processing order 2
MDC: [ orderID=456 ] INFO  - 200 OK: GET - /orders/456/item/789 in 1ms

In this simple example, you could extend the extractPathParameter function, but this becomes error-prone and tedious in a real-life scenario with 30+ routes and many different path parameters. It is easier to find a solution that knows the path parameters directly and does not depend on the URL structure.

So it is time to write our own Ktor plugin. The idea is simple: intercept the pipeline at a point where path parameters and their values are known, add them to the MDC, and then let the pipeline continue.

class PathParameterMDCPlugin {
    companion object Plugin : BaseRouteScopedPlugin<Unit, PathParameterMDCPlugin> {
        override val key = AttributeKey<PathParameterMDCPlugin>(”MdcPlugin”)

        override fun install(
            pipeline: ApplicationCallPipeline,
            configure: Unit.() -> Unit,
        ): PathParameterMDCPlugin {
            val plugin = PathParameterMDCPlugin()

            pipeline.intercept(ApplicationCallPipeline.Call) {
                call.parameters.forEach { key, values ->
                    MDC.put(key, values.joinToString())
                }
                withContext(MDCContext()) {
                    proceed()
                }
            }
            return plugin
        }
    }
}

After installing this plugin on the routes, we automatically get an MDC entry for each path parameter:

routing {
    install(PathParameterMDCPlugin)
...
}

MDC: [ itemId=123 ] INFO  - Processing request for item
MDC: [ itemId=123 ] INFO  - 200 OK: GET - /items/123 in 407ms
MDC: [ orderId=456, itemId=789 ] INFO  - Processing order 2
MDC: [ orderId=456, itemId=789 ] INFO  - 200 OK: GET - /orders/456/item/789 in 2ms

There is one final issue. We get itemID in the MDC for /items, but item for /orders. The fix is a one-time cleanup: unify the path parameter names. Unlike changing URLs, for example renaming …/item to …/itemID to satisfy CallLogging, renaming path parameters does not break clients. It is tedious work, but safe and done once.

With that, you now have everything needed to enrich your log messages with just a few lines of code.

The nice part is that most log aggregators, such as Graylog or Datadog, can index the MDC. If you use Datadog, you can simply query @orderID:123 to retrieve all logs for a given orderID.

Final Thoughts

I hope you found this useful. Having the MDC filled more or less automatically helped me a lot when finding the right log messages and filtering out unrelated ones while debugging a problem.

As a side note, neither Claude nor Junie came up with a robust solution for the inconsistent API path problem. Claude even struggled to create a working Ktor plugin without relying on deprecated APIs. So, Claude, if you are reading this, this is how it is done.

For transparency: I used ChatGPT to fix spelling and grammar issues.

As the year comes to an end, I want to reflect on my AI journey and how it gradually became part of my day-to-day work. My experience with GitHub Copilot dates back to mid-2023, when it gave me an early glimpse of AI-assisted development, without yet feeling transformative.1

Early Experiments and Mild Disappointment

I wrote about Copilot some time ago. Back then, I was kind of disappointed. Especially in IntelliJ, Copilot was merely an autocomplete on steroids. It was useful, no question, but it was far from groundbreaking.

I kept Copilot active in my IDE back then but did not see any amazing benefits or productivity gains. Throughout 2024, I mainly used my former employer’s own GPT version, which was not as good as ChatGPT but comparable. The workflow was the age-old “copy code into the prompt and ask for stuff.” Results were better, but it was not an efficient workflow.

2025 was the first year I used AI exhaustively in my day-to-day work. Due to the generous AI policy at my workplace, I got to try a lot of tools that year.

It started by ditching Copilot in favor of JetBrains AI. To me, they were similar, but JetBrains AI is way better integrated into IntelliJ, and the results are overall better (I still love the commit message generation function).

I continued to use ChatGPT and added Gemini to the mix. However, I do not ask them anymore to write tests, find bugs, or the like, although they have gotten way better at this kind of prompt. I use them as a sounding board for more abstract topics, such as architecture questions. You need to prompt them carefully, but in the end they are like the coworker who read all the blog posts. I would not base my decisions solely on a chat with them, but they provide good starting points. I also like to paste long stack traces into them, as they are faster at finding the important spots and often provide good solutions for more generic problems.

When AI Started to Actually Change My Work

This is all nice, but I still did not experience any really big improvements in my day-to-day work. The tools definitely save me time overall. However, I was able to produce code faster once I started using Junie, which became available in mid-2025 for us. To be honest, at first, I had the preconception that it was just some vibe coding junk. And vibe coding is dead, isn’t it?

But there’s no point in being stubborn. “We’ve always done it this way” is the dumbest argument there is. You have to try new things before you can hate them. And I hated it. At first.

I did not use Junie in my production code straight away. I tried to really vibe code. I wanted to do things I was not capable of doing because I lacked the knowledge, so I decided to create a simple IntelliJ plugin. I know Kotlin and Java, but I have no experience creating IntelliJ plugins. The goal was simple: a plugin that changes the text color in the file tree based on regex. For example, if a file matches .*Document, its name should be yellow. Junie quickly delivered convincing code, including a dialog box in the settings where you could configure the color and RegEx. But in the end, it didn’t work; the text always remained black. After two hours, I gave up in frustration. In retrospect, that was premature.

I then used Junie in a domain I am more knowledgeable about, and the results differed vastly. Junie was able to produce meaningful test suites following the general patterns already present. It could also create new functionality and alter existing functionality. The quality was good overall, but not perfect.

But one thing bugged me: it was painfully slow. Strikingly, it seemed slow even on easy tasks, such as modifying an enum. For a few days, I tried to use Junie for literally everything. Through a lot of trial and error, I learned how to get the most out of it. As a result, most of my work consisted of 30% Junie and 70% code I wrote myself, including modifications to Junie’s code.

Junie and I had grown into a happy relationship, until Jolene Claude appeared. Sorry Junie, but Claude is everything you are, just better. From day one, I was able to use Claude more efficiently than Junie. It is faster, and the results are better.

Don’t get me wrong: even Claude produces bullshit from time to time. I would never trust AI’s results blindly. But given a good prompt, the amount of corrections I had to apply to Claude’s code was minimal. In the end, the ratio shifted to 70% Claude’s code and 30% handwritten, including corrections and modifications.

Where This Leaves Me

So this is the state of the union for me currently. I can churn out code faster without compromising quality. From an economic point of view, that’s perfect. But is it still fun? To be honest, yes and no2.

I still love my job, even with the shift it has made due to AI. Writing code was never the focus of software development, at least not once you have some years of experience. Writing code was definitely the focus of my early years, which is probably why graduates and junior devs have a hard time now.

Reading, understanding, and debugging code (and maybe changing 1–2 lines after that) are far more common. This has increased even further with AI. Now I feel more like a pull request review machine than a coder. The number of actual pull requests opened on GitHub has increased, since people can write code faster. At the same time, I am constantly doing mini reviews on the code Claude produces. Code reviews are now my main focus (for the coding part) of my job.

Not sure how to feel about that, to be honest.

With that being said, I thank you for reading my short end-of-year AI summary. I wish you a happy new year (in which I hope to be able to write more again).

So long!

Chaos is part of a developer’s life. Priorities shift, tasks multiply, and the sense of being “done” is rare. I had planned to publish one post per month in 2025, yet here we are—only one post in February so far.

Not because I ran out of ideas. I wanted to write only quality posts, things that felt worth sharing, like my earlier one on Kotlin’s Result that came out of a big refactor. Then I started a new job, and life quickly filled with busyness.

That small story is just another example of how unpredictable and messy this work can be. Which is why I want to share the habits that help me keep a bit of sanity in the middle of engineering chaos.

The Unavoidable Morning Routine ☕️

I try to do this every morning: sit down, look at my calendar, and write down the three most important tasks of the day. The goal is to bring my brain into “work mode.” Combine this with a coffee, and voilà—you’ve got a simple morning routine.

If I recall correctly, I got this idea from the book Hyperfocus.

Morning routines are trendy right now. You’ll find thousands of videos of people—famous or not—explaining why their morning routine is the best one ever. To be honest, sometimes I wish I had one of those strict one-hour routines with sports, meditation, and everything else. But reality bites: I don’t want to wake up before everyone else just to cram all that in, and I simply don’t have the time every morning.

Still, these 10 minutes of focus help me immensely to start the day right.

The Brain Dump 🧠

If you’re familiar with Cal Newport, you might know his concept of a “shutdown routine”: closing open loops by moving tasks into a trusted system so they don’t linger in your mind after work.

Open loops are everywhere in software engineering. Some tasks—like reproducing a bug—can take minutes, hours, or even days, and during that time they stick in your head. I often found myself thinking about these things well into the evening.

My solution is simple: an end-of-day brain dump. I jot down, often unstructured, everything that comes to mind in my notebook:

• What I accomplished today

• What went well and what didn’t

• Any disagreements or frustrations

• Thoughts on tricky tasks or ongoing bugs

Some days, it’s just a couple of bullet points. Other days, it can fill several pages. The goal is always the same: get everything work-related out of my head.

If something pops into my mind later, I write it down immediately. This has significantly reduced the sleepless nights spent replaying problem X over and over.

It also helps me transition from “work” to “private life”—a crucial boundary when working from home.

Leave Things Broken for Tomorrow 🛠️

We’ve all been in retros, trying to recall what we worked on over the past few weeks. Sometimes, we can barely remember what we did the day before.

To make things easier, I’ve adopted a small trick: I don’t stop work with everything perfectly neat. Instead, I deliberately leave a test failing or a piece of code unfinished.

That way, when I sit down the next day, I know exactly where to pick things up. It jump-starts my brain and makes it easier to re-engage with the task.

Have a Personal “Definition of Done” ✅

You’ve probably heard of “Definition of Done” (DoD)—a Scrum buzzword meant to ensure a user story is truly complete. In my experience, these team-wide DoDs often fade into obscurity.

But having a personal DoD can be a game changer. Mine applies to pull requests, and before opening one, I mentally check off a short list, for example:

• Are all tests green?

• Have I removed debug logs or quick hacks?

• Is this the simplest yet most effective solution?

• Did I test this on Dev?

• (If applicable) Did I check performance for new queries?

This feels less like Scrum jargon and more like the Pointing and Calling method—simple but effective.

Focus Time Calendar Blockers ✋

For me, early mornings are the quietest—and most productive—parts of the day. That’s when I tackle the hardest problems, aka “eat the frog.”

To protect this time, I’ve blocked two slots in my calendar:

• 08:30–10:00

• 10:30–12:15

These are reserved for deep work. I try to protect them as much as possible, but I’m not dogmatic about it—if a meeting really needs to happen then, I’ll allow it.

The first block begins right after my short morning routine. It’s consistently my most productive window of the day.

Conclusion

One of the most important lessons I’ve learned is that there will always be open tasks. You are never truly “done.” That’s why it’s essential to have tools and habits that help you stay sane amidst the chaos.

In this post, I shared some of mine. Hopefully, they inspire you—or at least give you a starting point for developing your own.

For further reading:

• Deep Work by Cal Newport (Shutdown, Time Blocking)

• Hyperfocus by Chris Bailey (Three Most Important Things)

• Unfortunately, I can’t find the original post where I first read about “Leave things broken for tomorrow.”

If this post sparked a thought, I’d love to hear from you—drop me a line or leave a comment 💬. Always curious about how others tackle the chaos!

I had set a goal to release a blog post each month, but unfortunately, I missed January ⏳. Life and work sometimes get in the way, and I appreciate your patience. I’m excited to share this deep dive into a topic that has been on my mind for a while: why Kotlin’s Result type is not the best choice for error handling in application code. I hope you find it insightful—let’s dive in!

A couple of days ago, I had a discussion about Kotlin’s Result type vs. arrow-kt's Either. There are upsides and downsides to both, but I am a big fan of Either over Result, mainly because I think the design of Result comes with some flaws. Before we discuss these flaws, let’s take a step back and look at Result and why one might want to use it.

Understanding Result

Result is a class from the Kotlin standard library. The KDoc states that it is:

A discriminated union that encapsulates a successful outcome with a value of type T or a failure with an arbitrary Throwable exception.

In other words, we can use Result in places where we would throw an exception, with the difference that returning a Result will not change the control flow as a thrown exception does.

Take a look at this example:

fun unsafeParseInt(str: String): Int {
    return str.toInt() // This throws NumberFormatException for invalid input
}

fun main() {
    val numbers = listOf("1", "2", "abc", "4")

    val parsedNumbers = numbers.map { unsafeParseInt(it) } // 💥 CRASH on "abc"
    
    println(parsedNumbers)
}

If we use a function that throws an exception inside a collection operation like map, it can cause an unexpected crash. We could mitigate that by returning null instead of throwing an exception, but this weakens our API if there might be several different causes for that Exception. Let’s see how Result can help us here:

fun safeParseInt(str: String): Result<Int> {
    return try {
        Result.success(str.toInt())
    } catch (e: NumberFormatException) {
        Result.failure(e)
    }
}

fun main() {
    val numbers = listOf("1", "2", "abc", "4")

    val parsedResults: List<Result<Int>> = numbers.map { safeParseInt(it) }

    parsedResults.forEach { result ->
        result.onSuccess { println("Parsed: $it") }
              .onFailure { println("Error: ${it.message}") }
    }
}

Instead of letting the function throw, we return a Result. If everything worked fine, we can get the parsed number from the Result; otherwise, the Result will contain the exception. We can even write this more concisely:

fun safeParseInt(str: String): Result<Int> = runCatching { str.toInt() }

This behaves the same as the try-catch version above.

So far, so good. Result might be overkill for this example, but it has its uses. Think about API calls, which might fail regularly.

Issues with Result

The most significant issue for the described use case with Result is its signature:

@JvmInline
value class Result<out T> : Serializable

As you can see, Result only has one generic parameter, yet it covers two cases: success and failure. In case of failure, Result cannot tell you exactly what went wrong, as it only returns a Throwable. Why is this bad? Going back to the API call example, you definitely want to know what went wrong. Is the server not reachable? Did you provide an incorrect payload? Were the credentials invalid? Sure, you can use when expressions to check error types, but you might never know if you missed a case.

Because Result only explicitly knows the success type, it does not provide a meaningful map operation for the failure case. Additionally, Result lacks a flatMap function, making it cumbersome to chain multiple Result values.

Finally, failures in Result are always Throwable instances, which implies that something unexpected happened. However, this is not always the case, especially in the domain layer. Validation errors, for example, are not unexpected. They are not exceptions; they are the rule and can be modeled as domain classes. Result does not support this.

Why is Result designed this way? Because it was not designed to be used in application code. The Kotlin KEEP proposal specifically states:

The Result class is designed to capture generic failures of Kotlin functions for their latter processing and should be used in general-purpose API like futures, [...] The Result class is not designed to represent domain-specific error conditions.

Alternative: Either from Arrow

There are some alternatives to Result, but first, let’s discuss what to do if your code already uses Result. The Arrow library provides some utilities to make Result more usable in your code. First, it adds a flatMap function, making chaining Result instances easier. For more advanced cases, Arrow includes an implementation of its Raise API for Result.

fun result1(): Result<String> = TODO()
fun result2(): Result<String> = TODO()

fun workWithResults(): Result<String> = result {
    val string1: String = result1().bind()
    val string2: String = result2().bind()
    string1 + string2
}

Here, bind returns the success value if available or propagates the failure. If result1() returns a failure, workWithResults() will also return that failure, and result2() will never be called.

But this is just a workaround. The real solution is Either.

Why Either?

Either is not an invention of Arrow; it is a general-purpose solution from functional programming. Unlike Result, Either does not limit failure cases to Throwable. It has two generic types, one for success and one for failure:

object NotANumber // typed error

fun parseIntSafely(str: String): Either<NotANumber, Int> = try {
    str.toInt().right()
} catch (e: NumberFormatException) {
    NotANumber.left()
}

Either is a more flexible alternative to Result, as it allows defining a specific failure type instead of relying on Throwable. It follows a functional approach and supports useful operations like map, flatMap, and fold, making error handling more structured and predictable.

While Either is powerful, using it effectively requires understanding functional programming principles, which is beyond the scope of this post. If you want to dive deeper, check out Arrow’s documentation.

Rolling Your Own Solution

If Either feels too abstract, you can roll your own Result-like classes. However, I recommend this only for smaller applications, as you may end up reimplementing what Either already provides. For our string parsing example, it could look like this:

sealed class ParseResult {
    data class Success(val value: Int) : ParseResult()
    data object NotANumber : ParseResult()
}

fun parseIntSafely(str: String): ParseResult = try {
    ParseResult.Success(str.toInt())
} catch (e: NumberFormatException) {
    ParseResult.NotANumber
}

Verdict

If you can, avoid Result in your application code. It is not designed for domain-specific error handling. Rolling your own solution is feasible for small projects, but for medium to large applications, Either is the better choice.

That being said, it's always worth considering whether you need something like this at all. If you're dealing with a simple success-or-failure scenario, returning null for failure might be a reasonable and lightweight alternative. This pattern is already used in many Kotlin standard library functions, such as toIntOrNull, which:

Parses the string as an Int number and returns the result or null if the string is not a valid representation of a number.

Often, the simplest solution is the best.

What do you think about Result? Do you use it extensively in your code? Let me know in the comments below! Thanks for reading! 😊

Hey all 👋, it’s Mirco here with a quick update on my writing plans.

Long-time readers may have noticed that my publishing rhythm has been quite irregular this year, with bursts of weekly posts followed by month-long periods of silence. Reflecting on this past year’s challenges, I’ve rethought how to approach my writing schedule moving forward.

I love writing blog posts, but I must admit that, for now, I can’t match the high output of others on Substack. Writing posts on a weekly or bi-weekly basis didn’t work for me; it wasn’t a sustainable rhythm.

Therefore, my plan for 2025 is to publish at least one high-quality, long-form post each month. I plan to release them at the end of each month. By focusing on one valuable post each month, I can concentrate on creating deeper, more meaningful content. I may sprinkle in shorter posts now and then, but I’m publicly committing to twelve posts per year for some added pressure.

Thank you all for reading my articles and especially for your comments and other reactions.

I plan to release the first post in the new schedule at the end of December—I hope you enjoy it. 🙏

Let me take you back to a time when I worked on an old internal product — let’s call it Garbo. Now, Garbo wasn’t glamorous. It wasn’t directly responsible for generating revenue. It worked in the background, keeping everything running smoothly. That is, until one day, boom — it failed1.

Our customers got seriously irritated. And once that happened, we had to drop everything to fix it.

The codebase? It had been neglected for years. There was duplicate code everywhere, unused features still floating around, and the worst part? Tons of special-case logic for every exotic scenario you could think of. Business logic was scattered between the UI and the backend, making it nearly impossible to figure out what was actually in use. Testing? Yeah, right. Trying to make sense of the system was like unraveling a huge knot of code spaghetti. Tests were 80% mocking and 20% actual tests. However, most of the time it was unclear if the test did something useful.

It took four senior engineers (including me) months to make Garbo maintainable again. We spent countless hours untangling it just to keep things running smoothly. Time that could’ve been used to build something new 🛠️ was spent cleaning up a mess that could have been avoided.

Ever since then, every time I tackle a bigger refactoring project, I remind my product owner: “You don’t want a second Garbo.” And trust me, you don’t.

What could have saved us from this nightmare? Refactoring. Every. Single. Day. 🛠️

Refactor Every Day ⚙️

Spending just 20 minutes a day refactoring can save you hours of frustration down the road.

Whether it’s simplifying the code you just wrote or revisiting something from the backlog, small improvements make your codebase cleaner, more maintainable, and easier to understand. Refactoring isn’t just about cleaning up — it’s about understanding your code on a deeper level, and that leads to writing better code in the future.

Even the best-written code will eventually need refactoring. Hotfixes, last-minute feature requests, and quick patches can clutter even the most carefully crafted code. No matter how great your code is today, it’s going to need maintenance as things evolve.

Not sure where to begin? Here’s a quick checklist to get you started during your 20-minute daily refactor:

• Simplify logic: Break down bloated methods into smaller, more focused ones.

• Rename variables: Make sure your variable names clearly describe their purpose 📝.

• Remove unused code: 🗑️ Get rid of any dead code that’s no longer needed.

• Reduce complexity: Simplify deeply nested conditionals or loops when possible.

• Use your tools: 🛠️ Leverage your IDE’s refactoring features, like automatic method extraction, renaming, and refactor hints. These built-in tools make the process faster and less error-prone.

Want to dig deeper? Here are some great resources:

• Refactoring.guru: If you’re wondering where to start, this site is packed with refactoring tips. It’s like a treasure chest of practical advice — start here if you’re feeling stuck.

• Working Effectively with Legacy Code: If you're battling a legacy codebase (like Garbo), this book is a must-read. Sure, it’s a bit dated, but the advice is solid and will help you tame the beast.

• The Usual Suspects: You’ve heard of them — Clean Code, The Clean Coder, and Clean Architecture. These books are classics for a reason and worth your time if you’re serious about writing better, cleaner code.

• Martin Fowler wrote about this 13 years ago.2

It Benefits Everyone 🤝

This isn’t just about you — regular refactoring helps the entire team. Cleaner code means fewer bugs , faster feature development, and a much easier time onboarding new developers.

It’s a long-term investment in the health of your codebase.

Make it a habit. Refactoring. Every. Single. Day. Encourage your team to do the same. Refactoring doesn’t have to be a solo activity — pairing up with a teammate or incorporating it into code reviews can help spread the practice across the whole team.

If you’re not refactoring regularly, your codebase is slowly becoming harder to manage. Start making small, daily improvements and see how much easier life gets — for you and your team. Your future self will thank you 😊.

Disclaimer: Take everything here with a grain of salt — this is just my opinion based on my experience. Every codebase is unique, and what works for one may not work for another. Always approach advice with critical thinking and adapt it to your context.

Thanks for reading! Now it’s your turn — commit to just 20 minutes of refactoring each day and start transforming your codebase. Your team and your future self will be grateful!

Leave a comment

Two approaches can help: removing parts of the system to reveal the issue (via negativa1), or building it up incrementally (via positiva). These methods, known as divide and conquer or incremental debugging, are essential tools for tackling complex problems. Let’s explore how they work using a simple event-driven system as an example.

Say you have an application that receives events, parses them, applies some business logic, writes to a database, and then publishes events.

Via Negativa (Divide and Conquer): Isolate by Removing Dependencies ✂️

TL;DR: Start with your full system and gradually remove parts to isolate the issue.

Via negativa, often known as divide and conquer, involves starting with your complete system and gradually removing parts. After each step, check if the problem remains, disappears, or changes.

• If the problem doesn’t change, the removed part is probably not involved.

• If the problem disappears, the removed part is likely the culprit, but continue adding the parts back one by one to confirm.

• If the problem changes, the removed part contributes to the issue.

This can be done at the code layer (classes, methods, lines of code) or component level (e.g., database, messaging).

For example, if you suspect the business logic is causing the issue, remove the event consumption component first, then the mapping logic. If the bug remains, you’ve eliminated those as causes.

If the problem persists, continue removing components like event publishing or database interactions. Adding logging or unit tests helps pinpoint the failure. Track key points or test individual methods for better visibility.

If the problem disappears after a particular step, repeat the process by focusing on the removed component to isolate the faulty code or functionality.

Via Positiva (Incremental Debugging): Isolate by Adding Parts 🔨

TL;DR: Start with a minimal project and gradually add parts until the problem appears.

Via positiva, or incremental debugging, works in reverse. You start with a minimal setup and incrementally add parts of your code until the problem surfaces.

This approach is great when adding a new feature that doesn’t behave as expected. There might be hidden interactions between the existing code and the new addition. For instance, if you're adding tracing and traces aren’t showing up, is the issue with the implementation, or is a dependency pulling in an incompatible version?

In cases like this, via negativa might be impractical. Instead, create a new service with just the essential parts: consume messages, log them, and publish events—nothing more. You can run this locally and debug step by step.

Adding parts back incrementally helps identify which component introduces the issue. This method builds complexity in a controlled way, making it easier to pinpoint interactions or dependencies that cause the bug.

By recreating the problem in a simplified environment, you can isolate the root cause and test fixes with minimal interference.

Wrapping Up 🎁

The approach you choose depends on the problem. If you’re troubleshooting an existing system, divide and conquer (removing parts) works well. For new features, incremental debugging (adding parts) helps you build complexity until the issue surfaces.

Both methods make complex problems easier to tackle.

Thanks for reading! I hope this helped you think through new ways to approach problem-solving in code.

Building on the concepts covered in the first part, we’ll explore how to manage replication lag, resolve write conflicts, and efficiently partition data across multiple nodes.

These topics are critical for ensuring that your system remains responsive, reliable, and scalable as it grows.

In this post, you'll learn about:

• ⏳ Replication Lag: Understanding delays in data synchronization and how to mitigate them.

• ⚔️ Write Conflicts: How to handle conflicts in multi-leader and leaderless modes.

• 🔧 Fail-over Strategies: Ensuring continuous availability during node failures.

• 🧩 Partitioning: Dividing data efficiently across multiple nodes to handle Big Data and avoid hotspots 🌶️.

Replication Lag ⏳: Understanding Delays in Data Synchronization

Replication lag occurs when replicas have stale data due to delayed updates, common in async setups. It can also occur in leaderless setups depending on the consistency settings.

Leaderless systems like Amazon DynamoDB can mitigate lag through quorum reads and writes. In most systems, lag is typically only a few seconds, but it can be longer depending on performance vs. consistency needs.

Basic considerations 💡:

• Some applications may not care about replication lag, depending on the use case.

• If you always need strong consistent data, use synchronous replication.

• A basic workaround: Divide reads into categories. For strong consistency, read from the leader; otherwise, read from replicas. For example, this is supported in Google Cloud Spanner.

Types of Consistency 🧱:

• Read-After-Write 📚: Ensures that the user who wrote x never reads an older version of x after its write. Suitable when users mainly write small portions, like their profile.

• Monotonic Reads ↗️: Ensures that if a user reads x at a given time, they won’t see older versions of x later. Often implemented by assigning the same replica to a user, e.g., based on a hash of the user ID. This method can fail if the assigned replica is down.

• Causal Consistency 🔗: Ensures users see data in the correct causal order, such as never seeing an answer before a question.

• Configurable Consistency Modes: Many databases allow you to adjust consistency levels, giving flexibility to meet specific needs. Examples include Amazon DynamoDB, Apache Cassandra, MongoDB, Cosmos DB, Riak, and Couchbase.

• Leaderless Systems 🤷‍♂️: Rely on additional processes to manage consistency and mitigate lag:

  • Read Repair 👩‍🔧: Fixes inconsistencies during read operations by updating stale replicas with the latest data.

  • Anti-Entropy Processes 🛠️: Periodically synchronize replicas to ensure they converge to the same data.

We will discover Consistency in the upcoming Cheat Sheet post in more detail.

As we’ve seen, replication lag is a manageable challenge with the right consistency models and strategies.

Write Conflicts ⚔️ and Their Resolution: Keeping Your Data Consistent

Write conflicts happen in multi-leader and leaderless modes when nodes accept writes independently. Resolving these conflicts is key to maintaining consistency across replicas.

How They Happen 🔍:

• Concurrent Writes 📝: Multiple nodes write to the same data simultaneously without coordination.

• Network Partitions 🌐: Nodes become temporarily disconnected and accept conflicting writes.

• Timing Issues ⏱️: Variations in the timing of updates can cause mismatched data versions across nodes.

Common Resolution Strategies 🚑:

• Avoiding Conflicts Altogether: Assigning a responsible leader for each record can help avoid conflicts, similar to a partitioned leader approach. This approach fails if the leader goes down.

• Last Write Wins ⏱️: Each write is timestamped, and the highest timestamp wins. Drawbacks include potential data loss if writes are overwritten.

• Merging Data 🫂: If feasible, merge conflicting writes. Often requires additional application logic.

• Conflict-Free Replicated Data Types (CRDTs) 🤖: Data structures designed to resolve conflicts automatically. Complex and less supported in many databases.

Align your conflict resolution method with your application's specific needs, whether prioritizing speed, simplicity, or data accuracy.

Failover 🔧: Ensuring Continuous Availability During Failures

Failover is the process of switching to a backup when a primary or follower node fails, ensuring high availability and minimal downtime in distributed systems. It covers both leader and follower nodes to maintain seamless operation.

• Failed Followers: Mostly easy to recover. After the follower becomes available again, it requests all data from a leader that has been written since the failure. This assumes that the follower was previously synchronized up to a specific point; longer downtimes or high data volumes can complicate recovery.

• Leader Failover 👑: More complex.

  • Automatic: The system promotes a follower to a leader.

  • Manual: A human decides who will be the new leader.

  • Split Brain 🧠: Must be avoided to prevent multiple nodes from incorrectly assuming the leader role simultaneously, which can lead to data inconsistencies.

  • Failure Detection: It is challenging to determine if a leader failed. Typically, a timeout is used; if the leader does not respond within a set number of seconds, it is assumed to have failed.

  • Consensus Algorithms 🤝: Used for leader election but are a topic on their own and out of scope here. Common algorithms include Raft and Paxos.

Handling write conflicts is crucial for maintaining consistency in multi-leader and leaderless setups. With these resolution strategies in mind, let’s move on to explore failover strategies

Partitioning 🧩: Dividing and Conquering Data

Partitioning divides data across multiple nodes using a partitioning key, essential for managing large datasets where a single machine can't handle all the data —common in Big Data scenarios.

How it works:

• Each record belongs to a partition (shard)

• Partitions are typically replicated for availability and fault tolerance.

• The goal is to distribute both data and load evenly across nodes, avoiding skewed partitions (uneven data or load) and hotspots 🌶️ (partitions bearing most of the load).

• Sometimes called Sharding

Key-Value Data Partitioning 🔑

• Partitioning by Key Range 📏: Assigns each partition a range of keys (e.g., A-D, E-F).

  

  • Ranges can vary in size and may be set manually or automatically.

  • ✅ Maintains sorted data, making range queries straightforward

  • ❌ Can lead to hotspots 🌶️ if many keys fall within narrow ranges.

• Partitioning by Hash Key 🎲: Uses a hash of the key to determine the partition.

  

  • ✅ Ensuring even distribution across nodes.

  • ✅ This gives well-balanced data, depending on the hash function.

  • ❌ Makes range queries inefficient due to random distribution.

Challenges and Solutions 🚧

• Reducing Hotspots 🌶️: Prefixing keys with numbers or other identifiers (e.g., 00-99) can spread them across multiple partitions, distributing the load more evenly. However, this complicates reads, requiring the application to merge data from various partitions.

• Secondary Indexes 📚: Secondary indexes don't naturally align with partitions since they're independent of the partitioning key. They can be handled by:

  • Local Index (Partitioning by Document) 📄: Each partition maintains its index.

    • Queries must hit all partitions as the client can't know where the data resides.

    • Optimizations like routing information or query hints can sometimes reduce the need to query every partition.

  • Global Index (Partitioning by Term) 🌐: The index is partitioned by the index keys themselves, allowing more targeted queries but complicating writes as they may span multiple partitions.

Rebalancing ⚖️: Maintaining Even Distribution

As data grows or nodes are added, partitions can become unbalanced, necessitating rebalancing to distribute the load evenly:

• Fixed Number of Partitions 📦:

  • Set a fixed number of partitions upfront, often more than the number of nodes.

  • New nodes take over some partitions; departing nodes return them.

  • Too many partitions increase overhead, while too few limit scalability.

• Dynamic Number of Partitions 🔄:

  • Partitions split if they exceed a size threshold or merge if they shrink too much.

  • Works well for key range partitioning but can cause frequent rebalancing during growth.

• Rebalancing Costs 💸:

  • Moving data across partitions can be resource-intensive.

  • Prefer manual rebalancing during low-traffic periods to minimize impact.

Key Takeaways 📌

• Replication Lag ⏳ is a common issue in asynchronous and leaderless setups, where replicas may have outdated data due to delayed updates, affecting data consistency.

• Consistency Models: Various consistency levels, such as Read-After-Write, Monotonic Reads, and Causal Consistency, address different needs and scenarios in distributed systems.

• Write Conflicts ⚔️ occur in multi-leader and leaderless modes due to concurrent writes, network partitions, or timing discrepancies, requiring strategies like "Last Write Wins," data merging, or CRDTs for resolution.

• Failover 🔧 ensures high availability by switching to backups during node failures. Leader failover is particularly complex and demands robust processes to avoid split brain scenarios and ensure seamless leader election.

• Partitioning 🧩 divides data across multiple nodes, essential for handling large datasets in distributed systems, and requires careful management to distribute data and load evenly, avoiding hotspots 🌶️.

• Rebalancing ⚖️ is crucial as data grows or when adding nodes, redistributing data to maintain balance and efficiency, with strategies for fixed or dynamic partition numbers to adapt to changing loads.

If you found this guide helpful, please share it and drop your thoughts in the comments below—I'd love to hear your feedback!

Leave a comment

In the upcoming post, I’ll pull together my notes on Consistency to create a new cheat sheet.

Scalability is a key non-functional requirement that can sometimes feel a bit elusive, especially when you're diving into system design for the first time. This cheat sheet pulls together essential concepts of scalability in a practical, opinionated guide.

In this first part, we'll focus on the basics of scaling and replication, setting a strong foundation for anyone looking to build or optimize scalable systems.

You'll learn about:

• 🏗️ Vertical vs. Horizontal Scaling: The pros and cons of scaling up versus scaling out.

• 👑 Leader-Based Replication: Delve into single-leader and multi-leader setups, understanding how they manage consistency and scale.

• 🤷‍♂️ Leaderless Replication: Explore how leaderless systems handle writes and manage conflicts through quorum reads and writes.

• 🔄 Replication Methods: Compare synchronous, semi-synchronous, and asynchronous replication to balance performance and consistency.

What is Scalability in the First Place? 🚀

Scalability is all about ensuring your system can handle increasing loads —whether doubling, tripling, or more—without a drop in performance or a steep rise in costs. It’s crucial for sustaining efficiency as demand grows.

There are two main strategies:

🏗️ Vertical Scaling

Vertical Scaling means adding more resources (CPU, RAM, disk space, etc.) to a single host. It's less favored these days due to several downsides ❌:

• Cost and Performance Imbalance: Costs can rise disproportionately with performance gains.

• Physical Limits: There's a ceiling to how much you can scale a single machine, constrained by technology and physics.

• Downtime: Upgrades, like adding a new CPU, may require downtime.

• Limited Flexibility: Scaling down isn't straightforward, making it hard to respond to decreased loads.

🌐 Horizontal Scaling

Horizontal Scaling involves adding more hosts and distributing the load across them. It has several advantages ✅ over vertical scaling:

• Cost-Effectiveness: You can use off-the-shelf components (e.g., deploying 24 machines with 4-core CPUs instead of a single 96-core machine).

• Flexibility: It can adapt to traffic increases and decreases.

• Automation: Scaling can be fully automated, especially in cloud environments.

However, horizontal scaling also introduces complexity, such as managing distributed storage and computing, and handling increased network traffic.

In the age of cloud computing, horizontal scaling is often the preferred approach. Everything mentioned in this post will focus on horizontal scaling and its challenges.

Replication 👯‍♂️: Keeping Your Data Available

Replication is a key approach in horizontal scaling, where the basic idea is to copy data across multiple nodes. This setup offers several advantages ✅:

• Load Distribution: Each node only handles a fraction of the requests, balancing the workload.

• Fault Tolerance: If one node fails, requests can be redirected to another, improving system resilience.

• Reduced Latency: Clients can connect to geographically closer nodes, which reduces latency.

• Increased Data Safety: With multiple copies, the risk of data loss decreases.

However, replication introduces new challenges ⚠️:

• Increased Complexity: The overall system becomes more complex, both in terms of architecture and data management.

• Network Dependencies: Many operations now rely on network communication, adding potential points of failure and latency.

• Data Consistency: Keeping data synchronized across nodes is a major concern—should you strive for perfect consistency, or is eventual consistency acceptable?

Different replication approaches can be viewed through the lens of these challenges. Some setups designate specific nodes to manage consistency, leading to single-leader or multi-leader replication. Alternatively, leaderless replication takes a different approach altogether.

There’s also another axis to consider: how nodes communicate with each other. This gives rise to synchronous, semi-synchronous, and asynchronous replication models.

Leader-Based Replication 👑

Leader-based replication scales systems by designating leaders to manage write operations, which are then propagated to followers. This approach balances load, improves read performance, and enhances fault tolerance but also presents challenges related to consistency and failure management.

How It Works:

• Leaders and Followers:

  • The Leader 👑 handles all write operations and ensures changes are replicated to other nodes.

  • Followers receive updates from the leader and typically serve read requests, helping to distribute the load.

• Replication Methods:

  • Synchronous Replication: Writes are only considered successful when all followers have applied them, ensuring strong consistency but potentially increasing write latency.

  • Semi-Synchronous Replication: A write is successful when a subset of followers (often just one) applies the change, offering a middle ground between consistency and performance.

  • Asynchronous Replication: Writes are successful as soon as the leader processes them, with followers catching up later, improving performance but risking consistency issues due to replication lag.

✅ Advantages of Leader-Based Replication:

• Improved Scalability: Distributing read operations across multiple followers can significantly increase the system's ability to handle larger loads.

• Fault Tolerance: Followers can act as backups and be promoted to leaders if the current leader fails, providing resilience against node failures.

• Reduced Latency: By directing read operations to followers closer to the clients, latency can be reduced, enhancing user experience.

❌ Downsides of Leader-Based Replication:

• Replication Lag: Particularly with asynchronous replication, followers may not always be in sync with the leader, leading to stale reads and inconsistencies.

• Single Point of Failure: In single-leader configurations, the leader is a critical failure point, which can halt write operations until a new leader is elected.

• Conflict Resolution: In multi-leader setups, concurrent writes on different leaders can lead to conflicts, requiring strategies to resolve discrepancies between nodes.

• Split Brain: Network issues or failures can cause multiple nodes to incorrectly assume the leader role, resulting in data inconsistencies.

Single-Leader Replication 🥇

In this scenario, there exists exactly one leader.

✅ Advantages (compared to multi-leader):

• Simplicity and Predictability: The architecture is straightforward, making it easier to implement, debug, and reason about system behavior.

• Consistency Management: Having a single leader ensures a clear source of truth for data writes, simplifying consistency management.

❌ Downsides (compared to multi-leader):

• Single Point of Failure: The leader is a critical component; if it goes down, writes are paused until a new leader is elected, which can cause downtime.

• Scalability Limits on Writes: Since all writes funnel through a single leader, there's an upper limit to write throughput. Especially with synchronous replication, where every follower must confirm the write, leading to higher latency.

Multi-Leader Replication 🤝

Here, we can have an arbitrary number of leaders. The number of leaders also may change anytime.

✅ Advantages (compared to single-leader):

• Enhanced Write Availability: Writes can be accepted by multiple leaders, reducing the dependency on any single node and increasing write availability.

• Better Geographical Distribution: Leaders in different locations can handle writes closer to their respective regions, reducing latency for distributed users.

• Improved Fault Tolerance: If one leader fails, other leaders can continue to handle writes, reducing the impact of node failures on the system's availability.

❌ Downsides (compared to single-leader):

• Conflict Resolution Complexity: When the same data is written concurrently on different leaders, conflicts can occur, requiring complex conflict resolution strategies.

• Increased Operational Complexity: Managing multiple leaders adds complexity to the system, including ensuring all leaders remain consistent and coordinating writes between them.

• Split Brain Scenarios: Multi-leader setups are more susceptible to split brain issues during network partitions, where isolated leaders may independently accept writes, leading to data divergence.

Leader-based replication improves scalability but has drawbacks like single points of failure. Leaderless replication offers an alternative, with different trade-offs and solutions.

Leaderless Replication 🤷‍♂️

As the name suggests, in leaderless replication, there are no designated leaders—every node can accept writes, effectively making every node a leader.

How It Works:

• Quorum Concept:

  • Clients perform read and write operations on multiple nodes, using a concept called quorum:

  • With n total nodes, clients write to w nodes and read from r nodes.

  • All data is versioned with timestamps or version numbers.

  • For consistency, w + r > n ensures reads always include the latest version.

  • Example: With n = 5, w = 3, and r = 3, writing to three nodes means two might have outdated versions, but reading from three nodes ensures at least one has the current version.

  • 

• Failure Tolerance:

  • If w < n, the system remains operational even if some nodes fail (e.g., with n = 5 and w = 3, up to two nodes can fail without impacting availability).

  • Similarly, if r < n, the system can tolerate node failures during reads.

✅ Advantages (compared to leader-based replication):

• No Leader Failover Required: Eliminates the need for leader election and failover processes.

• High Resilience: Continues functioning even with multiple node failures.

❌ Downsides (compared to leader-based replication):

• Complex Conflict Resolution: Handling write conflicts is more complex since all nodes accept writes independently.

• Increased Latency: Quorum reads and writes can introduce additional latency due to interactions with multiple nodes.

Leaderless replication promotes resilience with no single point of failure but complicates conflict resolution and may increase latency. It's ideal for high-availability needs but requires careful management of data consistency.

Sync vs. Async Replication 🔄: Balancing Consistency and Performance

Replication modes—synchronous, semi-synchronous, and asynchronous—affect how writes are handled across nodes, balancing consistency and performance. Choose based on whether you need immediate consistency or faster writes.

Synchronous Replication

Synchronous Replication is the simplest mode. In this setup, the leader waits until all replicas have executed the write.

✅ Advantages:

• Strong Consistency: All replicas contain the same data immediately.

• Easier Fail-over: Since all replicas are in sync, any replica can become the new leader without the risk of lost writes after a leader failure.

❌ Downsides:

• Clients must wait until all replicas complete the write

• Sensitive to network latency.

• One failing replica renders the system unusable for writes.

• Not practical in leaderless scenarios.

Use Cases: Best for critical applications where strong consistency is paramount, like financial transactions.

Asynchronous Replication

In Asynchronous Replication, writes are considered successful once the leader executes the write.

✅ Advantages:

• Lower Latency on Writes: Improves system responsiveness.

• High Availability: The system remains operational even if some nodes fail.

❌ Downsides:

• Replication Lag: Nodes can get out of sync (we will discuss Replication Lag in part 2).

• Risk of Data Loss: If a leader fails, unsynced writes may be lost.

Use Cases: Suitable for applications where write speed is critical, and occasional data staleness is acceptable, like logging, analytics, or social networks.

Semi-Synchronous Replication

Semi-Synchronous Replication aims to get the best of both worlds. One follower per leader is synchronous, while the rest are updated asynchronously.

• Partial Consistency: Provides "partial consistency" and reduces the risk of lost writes.

• Use Cases: Ideal for systems needing a balance between consistency and performance, such as content management systems or collaborative tools.

In summary, synchronous replication ensures strong consistency but can add latency, while asynchronous replication boosts performance with potential data lag. Choose based on whether your priority is consistency or speed.

Key Takeaways 📌

Here are the key takeaways to solidify your understanding of scalability concepts covered in this guide.

• Scalability is essential for systems to handle increasing loads efficiently without performance loss or disproportionate cost increases.

• Horizontal Scaling 🌐 is preferred in modern architectures, spreading the load across multiple nodes, offering flexibility and automation but adding complexity in distributed data management.

• Replication 👯‍♂️ improves data availability and fault tolerance by duplicating data across multiple nodes, balancing load, and reducing latency for geographically distributed clients.

• Leader-Based Replication 👑 involves leaders managing writes and followers handling reads, offering simplicity and consistency but posing risks of single points of failure and potential conflicts in multi-leader setups.

• Leaderless Replication 🤷‍♂️ allows any node to handle writes, enhancing resilience but complicating conflict resolution and increasing latency through quorum-based reads and writes.

• Replication Modes 🔄 (Synchronous, Semi-Synchronous, Asynchronous) balance trade-offs between consistency and performance; synchronous offers strong consistency, while asynchronous favors performance at the risk of data lag.

If you found this guide helpful, please share it and drop your thoughts in the comments below—I'd love to hear your feedback!

Stay tuned for Part 2, where we’ll explore replication lag ⏳, write conflicts ⚔️, failover strategies 🔧, and data partitioning 🧩.

Special thanks to Nina for her great proofreading and tips! 😊

Happy scaling, and see you in Part 2! 🚀

1. Injecting Syntax Highlighting into a String 🌈

2. Search Anywhere 🔍

3. Keyboard Shortcuts PDF 📄

If you are still on vacation, check out the previous vacation special issue! 🏖️😎

Enjoy reading and feel free to share your thoughts in the comments below. Don't forget to subscribe for more insights and share this post with your network! 📢

Injecting Syntax Highlighting into a String 🌈

It's not unusual to have SQL or JSON as Strings in your code. Whether for tests or in your repositories, you might have something like this:

If your language supports multi-line strings, like Kotlin, this is already quite helpful to keep the formatting. However, you lose all the syntax information, and IntelliJ cannot help you if something is wrong with your JSON.

This is where language injection comes into play. Just add a comment to your String:

Now IntelliJ treats the string as it would treat JSON in a .json file. It even points out any errors in the JSON.

You can inject almost any language IntelliJ supports in this way:

Adding a comment is just one way to enable language injection. Be sure to check out the documentation for more details!

What are your favorite InelliJ features? Share in the comments!

Leave a comment

Find Anything Fast: Using IntelliJ’s Search Anywhere 🔍

IntelliJ's Search Anywhere feature is an incredibly versatile tool designed to help you quickly find anything within your project or IDE. By pressing Shift twice, you open the Search Anywhere dialog, which allows you to search for classes, files, symbols, and even IDE settings and actions from a single interface.

This powerful search functionality provides instant results as you type, with IntelliJ smartly prioritizing the most relevant matches. Whether you're looking for a specific method, navigating to a file, or trying to access a particular setting, Search Anywhere simplifies the process by consolidating all searches into one place.

Personally, I find this feature invaluable for tasks where I haven't memorized the shortcuts since I don't use them frequently.

Keyboard Shortcuts PDF 📄

IntelliJ's Keyboard Shortcuts PDF is a valuable resource for quickly learning and mastering the IDE's shortcuts.

By clicking on Help > Keyboard Shortcuts PDF, a PDF will open, organizing shortcuts by functionality and making it easy to find exactly what you need.

While plugins like Key Promoter X are excellent for learning shortcuts through practice, having the PDF open on a second monitor is incredibly useful for quick reference. I find it helpful to discover new shortcuts and refresh my memory on less common ones, making everyday tasks faster and more efficient in IntelliJ.

Wrapping Up

Thank you for exploring these productivity-boosting features of IntelliJ with me. Here's a quick recap:

• Injecting Syntax Highlighting into a String 🌈: Add syntax highlighting to your strings with a simple comment to improve code readability and error detection.

• Search Anywhere 🔍: Press Shift twice to search for classes, files, symbols, and settings all in one place, streamlining your navigation.

• Keyboard Shortcuts PDF 📄: Click to open a PDF with categorized shortcuts for quick reference and improved efficiency.

I hope these tips help you enhance your workflow and boost your productivity in IntelliJ. I'd love to hear your thoughts, questions, and feedback in the comments below. 💬 Don't forget to subscribe for more insights 🔔 and share this post with your network! 📢

Happy coding! 🚀

Leave a comment

Stay tuned for next week's post.

1. Music for Programming

2. Clean Code, The Movie 🎬

3. SQL Murder Mystery 🕵️‍♂️🔍

I'll be on vacation too, so the next issue will come out on the first of August. Wishing everyone a great time! 🏖️😎

Enjoy reading and feel free to share your thoughts in the comments below. Don't forget to subscribe for more insights and share this post with your network! 📢

Music for Programming (or any other task) 🎵

First, something for the ears. I often listen to music while coding, but I can't concentrate well if there are vocal parts. Instead, I quite enjoy listening to the tracks on musicforprogramming.net – well, the name says it all. The site currently has 71 episodes (I’d call them mixtapes) of music specifically designed to help you focus and get things done without distractions.

My favorite is episode 4, which features Com Truise. There’s even a podcast feed with all the episodes and an RSS feed available for easy access. So, next time you need some background music while working or studying, give it a try. You might find it just as helpful as I do. 🎧

What are your favorite tracks to code to? Share in the comments!

Leave a comment

Clean Code, The Movie 🎬

If you're looking to deepen your understanding of clean code principles, I highly recommend the video series featuring Robert C. Martin (Uncle Bob) live on stage.

Not everyone is a fan of his presentation style – it's a bit theatrical and sometimes polarizing – but the substance of his content is top-notch. In these videos, Uncle Bob dives into the nitty-gritty of what makes code clean, exploring topics like naming conventions, functions, and error handling.

You can find the series online, and it's a fantastic resource for anyone looking to improve their coding practices. Whether you're a seasoned developer or just starting out, the series offers practical advice that you can apply directly to your work. So, set aside some time, grab some popcorn, and get ready to learn from one of the best in the business. 🍿💻

SQL Murder Mystery 🕵️‍♂️🔍

Looking for a fun and engaging way to sharpen your SQL skills? Check out "SQL Murder Mystery" at mystery.knightlab.com. This interactive website turns learning SQL into an exciting detective game where you use your SQL powers to solve a murder mystery.

The premise is simple: a crime has been committed, and it's up to you to analyze data and piece together the evidence to find the culprit. As you navigate through the investigation, you'll write and execute SQL queries to uncover critical information.

So, if you're ready to put on your detective hat and dive into some data sleuthing, head over to the site and start solving the mystery. It's a fantastic way to learn while having fun. Happy investigating! 🕵️‍♀️🗂️🔍

Wrapping Up

Thank you for reading this week's vacation special issue of Weekly Tech Tidbits. Here's a quick recap:

• Music for Programming 🎵: Enjoy distraction-free music to enhance your focus.

• Clean Code, The Movie 🎬: Learn clean code principles from Robert C. Martin.

• SQL Murder Mystery 🕵️‍♂️🔍: Solve a murder mystery while sharpening your SQL skills.

I'll be on vacation, so the next issue will come out on the first of August. Wishing everyone a great time! 🏖️😎

I'd love to hear your thoughts, questions, and feedback in the comments below. 💬 Don't forget to subscribe for more insights 🔔 and share this post with your network! 📢

Leave a comment

Stay tuned for next week's post.

Hi 👋, it's great to have you here for this git special issue of Weekly Tech Tidbits. In today's post, we'll cover three intriguing topics:

1. The git shortlog Command 📝

2. Setting a Global .gitignore 🚫

3. Linus Torvalds on Git 📹

Enjoy reading and feel free to share your thoughts in the comments below. Don't forget to subscribe for more insights and share this post with your network! 📢

The git shortlog Command 📝

The git shortlog command in Git is an excellent tool for summarizing commit histories, providing a concise overview of contributions by author and commit message. It groups commits by author, making it perfect for tracking contributions and generating release notes.

Using git shortlog, you can easily see who contributed what to the project. Here’s a basic usage example:

git shortlog <start commit>...<end commit>
git shortlog HEAD~25..HEAD

This command summarizes all commits in the specified range, grouped by author and commit message. Here’s a sample output:

  10  Jane Doe <jane@example.com>
       Fix login issue
       Update user profile page
       ....

   5  John Smith <john@example.com>
       Initial commit
       ... 

This detailed output is ideal for creating a changelog. You can also shorten the output to quickly see who contributed the most:

git shortlog -sne

This command adds the number of commits (-s), sorts by the number of commits (-n), and includes the email addresses of the authors (-e). Here’s how the output might look:

  10  Jane Doe <jane@example.com>
   5  John Smith <john@example.com>

git shortlog is perfect for understanding the overall contribution landscape. Use it to identify active contributors or summarize changes for a new release. 🎉

For more information, check out the Git documentation on git shortlog.

Set a Global .gitignore 🚫

I always used to accidentally add heap dump files (*.hprof) to my Git repositories. It was a hassle to remove them from GitHub once I pushed them accidentally. To avoid this, I set up a global .gitignore to ensure such files are never tracked in any of my projects.

Here’s how to set up a global .gitignore:

Create a Global .gitignore File:

touch ~/dotfiles/.gitignore_global

Configure Git to Use the Global .gitignore File:

git config --global core.excludesFile ~/dotfiles/.gitignore_global

Add Common Exclusions

Edit the ~/.gitignore_global file and add patterns for files you always want to ignore. Here’s an example:

# Java heap dumps
*.hprof

# macOS system files
.DS_Store

For a more comprehensive list of ignore patterns, you can find useful templates at GitHub's gitignore repository.

Setting a global .gitignore helps keep your repositories clean by automatically ignoring files that are not relevant to your projects.

For more information, check out the Git documentation on ignore files.

Linus Torvalds on git 📹

Linus Torvalds, the creator of Git, gave an amazing talk at Google in 2007 (yes, the year the iPhone came out!). As o commenters say, the video quality might be a bit dated (it's 240p), but the content is pure gold – "Content 8k" according to one viewer. In this talk, Linus discusses Git, then a young system at just two years old.

Lean back and enjoy this insightful trip back in tech time!

Wrapping Up

Thank you for reading this week's Weekly Tech Tidbits. Here's a quick recap:

• The git shortlog Command: Summarize and understand commit histories efficiently.

• Setting a Global .gitignore: Keep your repositories clean by ignoring irrelevant files.

• Linus Torvalds on Git: An insightful talk from the creator of Git.

I'd love to hear your thoughts, questions, and feedback in the comments below. 💬 Don't forget to subscribe for more insights 🔔 and share this post with your network! 📢

Changelog:

• A reddit user mentioned that it is bad practice to put stuff under ~ which is a valid point. I changed it to ~/dotfiles/

Leave a comment

Stay tuned for next week's post.

1. Master Your Learning Paths with roadmap.sh 🚀

2. Kotlin Gems: The Scan Function 💎

3. Google Cloud Region Picker 🌍

Enjoy reading and feel free to share your thoughts in the comments below. Don't forget to subscribe for more insights and share this post with your network! 📢

Share verbosemode

Master Your Learning Paths with roadmap.sh 🚀

Learning new things is hard, especially when you don't know what you need to learn. This is where roadmap.sh comes in. They provide visual graphs of learning content for a wide array of development topics, such as backend, frontend, system design, and more.

While the site itself offers limited content, it provides valuable links for deeper exploration of these topics. If you log in, you can even track your own progress.

This site is incredibly useful if you want to dive into a new topic 💡. Recently, they launched a feature to create roadmaps on anything via AI. While I'm not yet convinced by the quality (the hand-curated ones are much better), it’s fun to try out.

Kotlin Gems: The Scan Function 💎

The scan function in Kotlin is a hidden gem for processing collections, similar to fold but with a twist. While fold accumulates values to a final result, scan retains all intermediate results, perfect for tasks like running totals or cumulative distributions.

Using scan, you can track the progression of computations 📈. Here’s an example:

val donations = listOf(10, 15, 20, 25, 30)
val runningTotal = donations.scan(0) { total, donation -> total + donation }

This code snippet starts with a running total of 0, sequentially adding each donation and capturing each intermediate state along the way. The scan function generates the following list: [0, 10, 25, 45, 70, 100], providing a detailed view of the accumulation process.

While both scan and fold have a similar function signature, they serve distinct purposes. Fold returns only the final accumulated value, which is suitable for scenarios where you only need the end result. Here’s an example demonstrating fold:

val donations = listOf(10, 15, 20, 25, 30)
val totalDonations = donations.fold(0) { total, donation -> total + donation }
}

This results in 100, giving you the cumulative sum without the intermediate steps.

The choice between scan and fold depends on your needs: use scan for visibility into each step of the accumulation, and fold when only the final result matters.

Google Cloud Region Picker 🌍

Choosing the right region for your cloud services can be challenging. The most important factor is proximity to your users, but costs and carbon footprint are also factors to consider.

I was surprised to discover that Google Cloud services can have different costs depending on the region. For instance, running Cloud Run instances in europe-west1 is cheaper than in europe-west3. Additionally, not all services are available in every region—Cloud Scheduler, for example, is only available in four regions in Europe.

The Google Cloud Region Picker simplifies this process, helping you select the most suitable region for your needs.

By setting just a few parameters, you'll receive a list of the best-fit regions for your services. This tool makes it much easier than digging through extensive documentation!

Wrapping Up

Thank you for reading this week's Weekly Tech Tidbits. Here's a quick recap:

• Master Your Learning with roadmap.sh: Simplify your learning journey with visual roadmaps for various development topics.

• Kotlin Gems: The Scan Function: Utilize the scan function for tracking the progression of computations and gaining intermediate results.

• Google Cloud Region Picker: Select the most suitable region for your cloud services based on proximity, cost, and availability.

I'd love to hear your thoughts, questions, and feedback in the comments below. 💬 Don't forget to subscribe for more insights 🔔 and share this post with your network! 📢

Leave a comment

Stay tuned for next week's post.

In the last post of this series, we set up our first CloudRun service with a simple "Hello, World." Now, we're ready to take the next step.

In this post, you'll learn how to set up a private artifact registry and use it to deploy your own Docker image to CloudRun. 🚀

Prerequisites

I assume you followed the previous post or have equivalent knowledge. If you're unsure, you can always go back and review or get the previous code from GitHub.

You must have a Google Cloud account and project, as well as gcloud and Terraform installed and running. We’ll also add Docker to the mix this time. Chances are, you already have it installed, but if not, you can find instructions on Docker's website.

To check if Docker is set up correctly, run the following command in your terminal:

docker run hello-world

You should see something like this:

Hello from Docker!
This message shows that your installation appears to be working correctly.

Great, now let's dive into today's task: getting your own Docker 🐳 image up to CloudRun!

Artifact Registry

The Artifact Registry is Google’s product for storing artifacts, including (but not limited to) Docker containers within your project. While you don’t have to use it to deploy your images to CloudRun—DockerHub works just fine—I find it convenient to keep everything in one place.

Like CloudRun, Artifact Registry includes a free tier, allowing you to store up to 500 megabytes for free. If you exceed that, you’ll be charged $0.10 per gigabyte per month. Additionally, data transfers crossing regions incur charges, so we'll set up our registry in the same location as our CloudRun service to avoid extra costs. However, even if you do incur charges, the prices are very reasonable. 💰

To get started, we must first activate the Artifact Registry API. Open apis.tf and add the following:

resource "google_project_service" "artifact_registry_api" {
  project            = local.project_id
  service            = "artifactregistry.googleapis.com"
  disable_on_destroy = false
}

This will enable the Artifact Registry API for your project.

Next, create a new file called artifactregistry.tf and enter the following:

resource "google_artifact_registry_repository" "artifact_registry" {
  location      = local.region
  repository_id = "docker-repository"
  description   = "Docker Repository for CloudRun"
  format        = "DOCKER"

  cleanup_policies {
    id     = "delete"
    action = "DELETE"
    condition {
      older_than   = "3600s"
    }
  }
}

Here, local.region refers to the region we set up in constants.tf in the last post. This is also used in the CloudRun configuration. Since we're just experimenting, I've added a cleanup policy that removes each image after one hour to ensure we don't incur any unwanted costs.

That’s all for now. Run terraform apply to set up the artifact registry. In the output, you should see something like this:

...
  # google_artifact_registry_repository.artifact_registry will be created
  + resource "google_artifact_registry_repository" "artifact_registry" {
...
      + description      = "Docker Repository for CloudRun"
...
      + format           = "DOCKER"
...
      + location         = <your location>
      + mode             = "STANDARD_REPOSITORY"
...
      + project          = <your project>
      + repository_id    = "docker-repository"
...

      + cleanup_policies {
          + action = "DELETE"
          + id     = "delete"

          + condition {
              + older_than            = "3600s"
              + package_name_prefixes = []
              + tag_prefixes          = []
              + tag_state             = "ANY"
              + version_name_prefixes = []
                # (1 unchanged attribute hidden)
            }
        }
    }

Before we can publish to the artifact registry, we must authorize access. This can be easily done via:

gcloud auth configure-docker  <your location>-docker.pkg.dev

Let's go ahead and publish a Docker image. 🐳

Docker Image

Now we are ready to push a Docker image to our registry. If you don't have one on hand, you can use the small Flask application I wrote for this purpose. It has a GET and a POST endpoint, which we can use to verify that everything is working correctly.

Get the Python code and Dockerfile from GitHub. Then navigate to the directory where you downloaded the files and build the Docker image with the following command:

docker build <path to the files> -t cloudrun101

Next, we need to add a tag containing the location of the registry and the project ID:

docker tag cloudrun101 <your region from constants.tf>-docker.pkg.dev/<your project id>/docker-repository/cloudrun101:latest

Finally, we can push the image:

docker push <your region from constants.tf>-docker.pkg.dev/<your project id>/docker-repository/cloudrun101

If everything succeeded, you can check the registry in the web interface and see something like this:

Now we can use it in our CloudRun service.

Use the Image in CloudRun

We only need to change the image name in our cloudrun.tf file:

# Terraform configuration to create a Cloud Run service
resource "google_cloud_run_v2_service" "service" {
....
  template {
    containers {
      image = "${local.region}-docker.pkg.dev/${local.project_id}/docker-repository/cloudrun101:latest"
    }
  }

}

Run terraform apply to update your CloudRun revision with the new image. Note that I kept the public access as discussed last time for easier testing. Once terraform apply succeeds, you can test if the image works by opening <your-cloudrun-url>/hello in your browser.

This may not be the most useful CloudRun service, but it runs our own image from Artifact Registry! Great! 🎉

Wrapping Up

Congratulations! 🎉 You've set up a private artifact registry and deployed your own Docker image to CloudRun.

Remember to run terraform destroy to clean up your resources.

Stay tuned for the next post, where we'll connect Pub/Sub to CloudRun. Thanks for following along, and happy coding! 🚀

1. IntelliJ IDEA's New K2 Mode 💻

2. Hidden Kotlin Gems: The Windowed Function 💎

3. Future Directions for Kotlin: What's Next? 🔮

Enjoy reading and feel free to share your thoughts in the comments below. Don't forget to subscribe for more insights and share this post with your network! 📢

Share verbosemode

IntelliJ IDEA's New K2 Mode 💻

IntelliJ IDEA 2024.1 has introduced an optional K2 mode, utilizing the new K2 compiler for enhanced Kotlin support. This mode promises:

• Compatibility with future Kotlin features

• Improved code analysis stability

• Better IDE performance 🚀

However, as an Alpha release, it has notable limitations, including:

• Lack of Methods and Calls Hierarchy 🚫

• No support for Android projects

• No code analysis in .gradle.kts files

Despite its current limitations, K2 mode offers an exciting glimpse into the future of Kotlin development. IntelliJ IDEA is clearly making strides towards a more efficient and robust development environment.

While I wouldn't use it daily just yet, I'm eager to see how it evolves.

Kotlin Gems: The Windowed Function 💎

The windowed function in Kotlin is a hidden gem for processing consecutive elements in overlapping segments, or "windows." This function is perfect for tasks like calculating moving averages, smoothing data, or analyzing trends by creating sublists of a specified size, sliding over the original list with a specified step.

Using windowed, you gain a more granular view of your data 📊, detecting patterns and anomalies that an overall average might miss. Take a look at this example:

fun main() {
    val responseTime = listOf(72, 75, 78, 80, 77)
    val windows = responseTime.windowed(size = 3, step = 1)
    val trends = windows.map { window -> window.average() > 75 }
}

This code creates windows of 3 consecutive measurements, sliding one step at a time. The windowed function produces the following windows:

Windows: [[72, 75, 78], [75, 78, 80], [78, 80, 77]]

And checks if the average response time in each window exceeds 75, resulting in:

Trends: [false, true, true]

This approach is more helpful than averaging the entire list because it detects periods of higher response times that might be missed when looking at the overall average 📈.

For example, in the list of response times[72, 75, 78, 80, 77], the overall average is 76.4. However, this average hides the fact that there are consecutive higher response times (78, 80, 77). By looking at smaller windows of 3 elements each, you can identify these spikes and trends that an overall average would overlook.

Future Directions for Kotlin: What's Next? 🔮

I highly recommend watching Michail Zarečenskij's talk on "Kotlin Language Features in 2.0 and Beyond." Kotlin 2.0 introduces several cool features that enhance the language's expressiveness and developer productivity.

Looking ahead, Kotlin plans to introduce even more powerful features, making it a language of choice for many developers.

Wrapping Up

Thank you for reading this week's Weekly Tech Tidbits. Here's a quick recap:

1. IntelliJ IDEA's New K2 Mode: Promising but currently limited.

2. Hidden Kotlin Gems: The windowed function offers powerful data analysis capabilities.

3. Future Directions for Kotlin: Exciting features on the horizon to enhance language expressiveness and productivity.

I'd love to hear your thoughts, questions, and feedback in the comments below. 💬 Don't forget to subscribe for more insights 🔔 and share this post with your network! 📢

Leave a comment

Stay tuned for next week's post.

• Efficient Communication with the Minto Pyramid 📐

• Key Takeaways from Kat Gaines' Talk on Incident Communication 🎤

• Insights on Slow Productivity from Cal Newport's New Book 🐌📕

Enjoy reading and feel free to share your thoughts in the comments below. Don't forget to subscribe for more insights and share this post with your network! 📢

Share verbosemode

Efficient Communication with the Minto Pyramid

The Minto Pyramid is an effective framework for structuring written communication, particularly useful in incident communication.

The structure is fairly easy:

• Main Point (WHAT): State the incident and its immediate impact in 1-2 sentences.

• Key Information (WHY | HOW): Provide details such as the cause of the incident or steps being taken to resolve it.

• (Optional) Additional Details: Include further information that might interest specific stakeholders, like technical logs or timelines.

Here is an example incident communication using this principle:

Service X has been unavailable since June 12, 2024, affecting approximately 15% of our users.

Incident Overview

1. Cause: Our investigation indicates that the outage is likely due to a load balancer failure.

2. Actions Taken: A technical team is currently working to resolve the issue.

3. Estimated Resolution Time: We expect the service to be restored within the next 2 hours.

Additional Details

• Several restarts and configuration checks have already been performed to isolate the issue.

By using the Minto Pyramid, you can ensure your incident communications are clear, concise, and effective, providing essential information promptly and accurately.

Key Takeaways from Kat Gaines' Talk on Incident Communication 🎤

My example above may not be very useful for end customers. I am biased toward incident communication like this since I work on internal software that is not directly customer-facing, unlike my company's website. However, everyone also has internal customers, and most likely not all of them are techies.

This requires another level of incident communication. Kat Gaines from PagerDuty recently gave a short talk on this topic.

Here are the key takeaways:

1. Involve customer-facing teams as soon as possible: They know your customers and how to communicate with them (around minute three)

2. Have pre-defined templates that are easy to fill out: This will reduce the stress during an incident while keeping stakeholders and customers informed (around minute fifteen)

3. Give frequent and meaningful updates: Keep notification fatigue in mind(Around minute nineteen)

Kat's talk is only 25 minutes long, so it can fit into your lunch break. Alternatively, you can use the timestamps provided above to get the main gist.

Slow Productivity 🐌📕

I am an avid reader, and since 2021, also a listener of Cal Newport's ideas. I can warmly recommend everyone to read his famous book “Deep Work” and listen to his podcast.

A couple of months ago, he published his new book called “Slow Productivity”. Main topic of the book is that the definition of productivity in the knowledge sector is broken. He refers to this as “pseudo productivity”. Since there is no good way to measure productivity (as opposed to a factory worker who can be measured by produced “units” per hour), many knowledge workers use visible business as a proxy for productivity, e.g. by joining a lot of meetings and there like.

Cal wants to coin an alternative approach which he calls “Slow Productivity” as another way to see productivity in knowledge work. Slow Productivity is based on the three principles

1. Do Fewer Things: Focus on fewer tasks to achieve more meaningful accomplishments.

2. Work at a Natural Pace: Avoid the constant rush and instead find a sustainable rhythm.

3. Obsess Over Quality: Prioritize high-quality work over high-quantity outputs.

While I think the overall content of the book is just fine, I was a little bit disappointed. The book is full of anecdotes which act as anecdotal evidence for the principles, but they are far too long and frequent. On could easily reduce the page count of the book by 25% without loosing any of its meaning. Also, many of the ideas feel like a rehash of content Cal Newport already produced.

So if you are familiar with Cal Newports ideas, I’d suggest to read the Blink or another summary first.

Wrapping Up

Thank you for reading this week's Weekly Tech Tidbits. Here's a quick recap:

• Efficient Communication with the Minto Pyramid: Ensure clear and concise incident communications.

• Kat Gaines' Talk on Incident Communication: Involve customer-facing teams, use templates, and provide meaningful updates.

• Slow Productivity from Cal Newport: Focus on fewer tasks, work at a natural pace, and prioritize quality.

I'd love to hear your thoughts, questions, and feedback in the comments below. 💬 Don't forget to subscribe for more insights 🔔 and share this post with your network! 📢

Leave a comment

Stay tuned for next week's post.

• the concept of Disagree and Commit 🗣️✅

• how to keep your code tidy 🧼

• Typed Error Handling in Kotlin ⚠️🎥

If you missed the previous issue, find it here.

Enjoy reading and feel free to share your thoughts in the comments below. Don't forget to subscribe for more insights and share this post with your network! 📢

Share verbosemode

Disagree and Commit 🗣️✅

In any collaborative environment, disagreements are inevitable. However, the principle of "disagree and commit" provides a powerful attitude for maintaining momentum and unity.

The idea is simple: after thoroughly considering all viewpoints, a decision is made. Even if you initially disagreed with the chosen direction, once the decision is finalized, you fully commit to its implementation. You leave all your doubts behind and accept that your suggestion was not accepted.

However, this only works effectively if all opinions on the issue are heard and taken seriously prior to making the decision; otherwise, it can lead to entrenched conflicts, negative sentiments, and hidden rivalries.

It's important to work on yourself to adopt this attitude, cultivating humility and avoiding stubbornness to ensure the process is genuine and effective.

Several companies live by this principle, including GitLab, Netflix, and most prominently Amazon. Even if your company isn't as big as Amazon1, adopting this mindset can still drive significant improvements in team collaboration.

Keep your Code Tidy with Spotless 🧼

Maintaining a consistent code style is essential for noise-free merges and readable code, yet synchronizing these styles across a team can be a hassle. The key is to automate this process, allowing developers to focus on coding and not on manual formatting.

Spotless addresses this need by enabling you to write code without worrying about style inconsistencies until it’s time to merge. Spotless can be easily integrated to your build system as plenty plugins exist.

Here's how it works:

1. Define which formatters and linters you want to use (e.g. ktlint and diktat for Kotlin). This is only done once in your build system, so it is the same for everyone.

2. Write your code without focusing on style.

3. Before opening a pull request, run spotless apply to ensure your code adheres to the predefined style guidelines.

4. Finally, run the spotless check before merging to maintain consistency on the main branch.

This approach is challenging to implement 1:1 in a trunk-based development style, but you can use pre-commit hooks to automatically run Spotless before each commit, ensuring consistency throughout the development process.

By integrating spotless into your workflow, you ensure that your team’s code remains clean and consistent, enhancing both code quality and team efficiency.

Typed Error Handling in Kotlin 🎯🎥

Typed error handling is an alternative to the traditional try-catch method, offering a more structured way to manage errors in your code. This approach uses types to explicitly represent possible errors, enhancing code readability and maintainability by ensuring that all potential failures are clearly defined within the type system.

Kotlin is well-suited for typed error handling due to its type system and functional programming capabilities. Here is a short example of typed error handling:

// try-catch
fun parseNumber(str: String): Int {
    return try {
        str.toInt()
    } catch (e: NumberFormatException) {
        -1 // or some error handling logic
    }
}

// typed error handling
sealed class ParseError {
    object NotANumber : ParseError()
}

fun parseNumberTyped(str: String): Either<ParseError, Int> {
    return str.toIntOrNull()?.right() ?: ParseError.NotANumber.left()
}

For a practical and in-depth look at functional error handling in Kotlin, I highly recommend the video "Functional Error Handling – A Practical Approach" by Bas de Groot.

This talk provides a short and useful way to understand typed errors, addressing the problems they solve, how to implement them, and leveraging the Arrow library.

Wrapping Up

Thank you for reading this week's insights. To summarize, adopting 'disagree and commit' can enhance team unity, using Spotless ensures clean code, and leveraging typed error handling in Kotlin can make your code more reliable.

I'd love to hear your thoughts, questions, and feedback in the comments below. 💬 Don't forget to subscribe for more insights 🔔 and share this post with your network! 📢

Leave a comment

Stay tuned for next week's post!

CloudRun has become my go-to solution for deploying applications in the cloud due to its simplicity and scalability. It's an ideal choice for various use cases.

In this upcoming blog post series, I'll guide you on how to get started with CloudRun effectively. We'll begin with the quintessential "Hello, World," using Terraform to manage the infrastructure. You can get the complete code on GitHub.

But first, let's take a look at the basics of CloudRun and understand why it's such a powerful tool.

What is CloudRun? ☁️🏃‍♂️

CloudRun is a fully managed serverless platform on the Google Cloud Platform (GCP). It allows you to deploy containerized applications written in any programming language or framework. The platform automatically scales to handle incoming requests based on demand, making it suitable for a wide range of applications, from small-scale projects to large enterprise applications. CloudRun follows a pay-as-you-go pricing model, making it cost-effective.

You can deploy any Docker container to CloudRun. The platform operates on a request-based model, meaning it starts your container when an HTTP request comes in and processes it. If there are no incoming requests, CloudRun scales down to zero, resulting in zero costs for you.

While there are ways to keep containers alive without requests, the primary model of CloudRun is request-based. It is not designed for long-running operations, especially background processes. For such needs, Google Cloud offers another service called CloudRun Jobs, which is specifically tailored for long-running processes.

Setting Up the Environment

Setting up the environment for deploying applications on Google Cloud Platform (GCP) involves a few key components: a GCP account, the gcloud command-line tool, and Terraform for infrastructure provisioning. I assume that you already have a GCP account and a Google Cloud Project. I also take basic command line skills for granted.

We need to install two essential command line tools:

• gcloud is the primary command-line interface for interacting with GCP resources. Install gcloud by following the instructions provided in the GCP documentation.

• Terraform is an infrastructure as code tool that allows you to define and provision cloud infrastructure using a declarative configuration language. You can download and install Terraform from the official Terraform website. I recommend using a version manager like tfenv to install Terraform.

Now we are ready to go furthe!

A Quick Terraform Primer 🏗️

Terraform is a tool to manage infrastructure as code. Instead of manually clicking through a web interface to set up resources, Terraform utilizes HCL (HashiCorp Configuration Language), a YAML-like language, to describe what resources you need. HCL is a declarative language (like SQL): instead of describing how to set up your infrastructure, you write what you expect as the end result. So you define how your CloudRun service should look, and Terraform determines what to do and in what order.

For this, Terraform needs to keep a state. This is a plain text file where Terraform saves which resources it has created and details about them. Since Terraform relies on this state, it is important to never change resources created by Terraform without using Terraform.1 Although you could manually change resources and get away with it, it would confuse Terraform and cause issues later. Only do this if you are sure about the changes. Additionally, never cancel a Terraform operation, as it may result in a corrupted state. If you make a mistake, let Terraform complete the operation, correct the mistake, and then run Terraform again.

To interact with Terraform, it is sufficient to know these commands:

• terraform init: Initializes Terraform (similar to git init). Must be done only once.

• terraform plan: Shows what Terraform will do without actually performing any actions (dry run).

• terraform apply: Creates the resources.

• terraform destroy: Destroys all resources saved in the state.

Since this is not primarily a Terraform tutorial, I aim to balance best practices with simplicity. Check the section at the bottom for a discussion on what could be improved in a real-world scenario.

There are many great resources to learn Terraform, but this overview is sufficient for now.

Lastly, we must enable Terraform to create GCP resources on our behalf. Run the following command to create a credentials file for Terraform. It will prompt you to open a browser and log in with the account you want to use:

gcloud auth application-default login

Now can create our CloudRun service.

Hello, CloudRun!

Now that we have the necessary tools installed and initialized, we can move on to the exciting part—setting up and deploying our CloudRun service. In the following sections, we'll configure Terraform to create and manage our CloudRun infrastructure.

Initialize Terraform

Go to the folder where you want to store the Terraform configuration files and create a file called constants.tf to hold your project settings:

# constants.tf
locals {
  # The project ID where the Cloud Run service will be created
  project_id = "..."
}

Next, we must tell Terraform to load the Google Cloud provider so it can communicate with GCP. Create a new file called provider.tf:

# provider.tf
terraform {
  required_providers {
    google = {
      version = "~> 5.32.0"
    }
  }
}

provider "google" {
  project = local.project_id
}

Now, use terraform init to load the provider and create your workspace.

> terraform init                                                                                                                                                 

Initializing the backend...

Initializing provider plugins...
- Reusing previous version of hashicorp/google from the dependency lock file
- Using previously-installed hashicorp/google v5.32.0

Terraform has been successfully initialized!

Since we have all the tools ready, we can start setting up the CloudRun.

Adding CloudRun

Before you can use any service on GCP, you first must activate its API. Create a new file called apis.tf:

# apis.tf
resource "google_project_service" "run_api" {
  # references the value in constants.tf
  project            = local.project_id
  service            = "run.googleapis.com"
  disable_on_destroy = false
}

disable_on_destroy tells Terraform to keep the API activated even if you call terraform destroy.

Now create cloudrun.tf to add the actual service:

# cloudrun.tf
resource "google_cloud_run_v2_service" "service" {
  name     = "hello-cloudrun"
  location = "europe-west1"
  ingress  = "INGRESS_TRAFFIC_ALL"

  template {
    containers {
      image = "us-docker.pkg.dev/cloudrun/container/hello"
    }
  }

}

The location specifies where your service should be hosted. Choose a region that is close to your users. You can use this handy tool to select the region which suits you the most. Ingress describes how your service can be accessed. We choose to accept all traffic. You could also allow only load balancer traffic or only traffic from your internal GCP network, which we will explore in another post.

The template section describes how your service should look. Here, we only specify the Docker image and leave the rest with the defaults.

Next, create a file called outputs.tf to print the URL of the CloudRun service to the terminal after it is created:

# outputs.tf
output "cloudrun_url" {
  value       = google_cloud_run_v2_service.service.uri
  description = "The URL of the Cloud Run service"
}
 

We are all set now. Run terraform plan to see what will happen:

> terraform plan

Terraform used the selected providers to generate the following execution plan. Resource actions are
indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # google_cloud_run_v2_service.service will be created
  + resource "google_cloud_run_v2_service" "service" {
...
      + ingress                 = "INGRESS_TRAFFIC_ALL"
...
      + location                = "europe-west1"
      + name                    = "hello-cloudrun"
...
      + project                 = "premium-oven-425714-n3"
...
      + uri                     = (known after apply)

      + template {
...

          + containers {
              + image = "us-docker.pkg.dev/cloudrun/container/hello"
            }
        }
    }

  # google_project_service.run_api will be created
  + resource "google_project_service" "run_api" {
      + disable_on_destroy = false
      + id                 = (known after apply)
      + project            = "your_project_id"
      + service            = "run.googleapis.com"
    }

Plan: 2 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + cloudrun_url = (known after apply)

Looks good. Terraform will enable the API, create a CloudRun service, and then print its URL. Now, run terraform apply to create the resources:

> terraform apply


Terraform used the selected providers to generate the following execution plan. Resource actions are
indicated with the following symbols:
  + create

Terraform will perform the following actions:
...

Plan: 2 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + cloudrun_url = (known after apply)

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

google_project_service.run_api: Creating...
google_cloud_run_v2_service.service: Creating...
google_project_service.run_api: Creation complete after 4s [id=.../run.googleapis.com]
google_cloud_run_v2_service.service: Still creating... [10s elapsed]
google_cloud_run_v2_service.service: Creation complete after 10s [id=projects/.../locations/europe-west1/services/hello-cloudrun]

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

Outputs:

cloudrun_url = "https://hello-cloudrun-...a-ew.a.run.app"

It is possible that you run into an error:

Error: Error creating Service: googleapi: Error 403: Cloud Run Admin API has not been used in project … before or it is disabled.

It may some take some time to really enable the API. If this is the case, just wait a couple of minutes ☕ and run terraform apply again.

Great! If you now open the URL in the browser, you’ll see the following:

Bummer. Didn’t we allow all traffic via INGRESS_TRAFFIC_ALL? True, but here comes another handy feature of Cloud Run: secure by default. It allows traffic from the internet, but only authenticated traffic.

If you want to access your service in the browser, you can do two things. First is to provide a correct Authentication header in your request. To do this, you first must install a browser extension that allows you to set request headers. There are plenty to choose from depending on your browser.

To get a valid token, run gcloud auth print-identity-token in your terminal. This will print a valid OAuth token. Now, use your extension to add the Authorization header, and set it to the value Bearer <token from above>. If you now open the URL again with the header activated, you’ll see this:

So you did it! There’s your up-and-running CloudRun service! 🥳🍾

As mentioned above, there is a way to access the service without authorization. To do this, you must assign the role run.invoker to everyone. All users with this role may call your Cloud Run services. To do this, add the following to your cloudrun.tf and run terraform apply again:

# cloudrun.tf
...

resource "google_cloud_run_v2_service_iam_member" "public_access" {
  project  = google_cloud_run_v2_service.service.project
  location = google_cloud_run_v2_service.service.location
  name     = google_cloud_run_v2_service.service.name
  role     = "roles/run.invoker"
  member   = "allUsers"
}

Now, you (and everyone else on the internet 🧑‍💻!) can just open the provided URL. Obviously, you should use this with care.

We are done for now. 🎉 Call terraform destroy 💣 to delete the resources created. Next time, we will see how to deploy our own Docker images to CloudRun, using our own registry!

Wrapping Up

Congratulations on deploying your first CloudRun service! 🎉 You’ve taken a big step in leveraging the power of serverless infrastructure. Head over to GitHub to get the complete code!

As mentioned above, this is not a comprehensive Terraform tutorial. In the real world, you would do some things differently.

We saved the Terraform state in a local file. The state contains all information about what Terraform created in the cloud. In a real-world scenario, you would likely save the state in the cloud itself, ensuring that the latest version is accessible to every team member. Terraform self-manages a lock on the state, preventing concurrent modifications.

Thank you for reading! In the next article of this series, we will explore how to deploy our own Docker images to CloudRun using our own registry 📦. Stay tuned for more insights and tips. 🚀

I'd love to hear your thoughts, questions, and feedback in the comments below. 💬 Don't forget to subscribe for more insights 🔔 and share this post with your network! 📢

• Compound interest of mentoring 💡

• How to save time during git rebase ⏱️

• How to choose your tools 🪚

Enjoy reading and feel free to share your thoughts in the comments below. Don't forget to subscribe for more insights and share this post with your network! 📢

Share verbosemode

Compound interest of mentoring 💡

The more experienced you become, the more you see that knowledge compounds. Learning your first programming language is hard, but the second one is easier since you already understand loops, classes, and variables. After working with one cloud provider, switching to another is simpler because the underlying challenges, like role management, are similar.

However, your time is limited. As a senior developer, you may be faster than less experienced colleagues, but you also attend more meetings, leaving less time for actual implementation.

You can optimize your day by minimizing non-coding activities to boost your output, especially when deadlines are near. But a smarter approach is to enable others to increase their output. If you can help two people improve, you invest a small portion of your time to gain double the value for the team. If those people mentor others, the benefits multiply exponentially.

The math may be off, but the essence is clear: the more you help others improve, the more your team benefits.

If you lack time for intensive mentoring, consider these strategies:

• Write and Update Documentation: Help others solve problems independently with clear documentation.

• Write Detailed Comments in Code Reviews: Provide useful feedback and links to relevant documentation or articles.

• Host Regular Knowledge Sharing Sessions: Set up weekly or bi-weekly sessions where team members can share their expertise on different topics.

Investing time in mentoring boosts both individual and team productivity, creating a more efficient and skilled team over time.

If you want to read more on mentoring, I recommend this article by .

Save rebase time with fixup and autosquash ⏱️

There is a common pattern when you work with feature branches and pull requests. After receiving comments on your pull request, you add changes to the branch, leading to new commits. However, not each commit must be preserved and merged to the main branch. Especially smaller commits, like fixing a typo, are of no value. So what you typically do is:

1. Fix the typo in the commit.

2. Run git rebase -i <SHA>.

3. Manually reorder your commit.

4. Change the action to fixup.

5. Execute.

This process is a bit tedious, especially if you have to do it often. Another way is to use git commit --fixup.

Imagine this initial git history:

* 316e87f (HEAD -> master) add bar
* 226733c first commit  <-- commit with typo
* a9c145c initial

You want to add a commit between 226733c and 316e87f to fix it up later. You can add the commit using git commit --fixup 226733c, which leads to this history:

* 4da3895 (HEAD -> master) fixup! first commit <-- fixup commit
* 316e87f add bar
* 226733c first commit  <-- commit with typo
* a9c145c initial

It's important to note that the whole commit message was added by git itself. You can now call git rebase -i --autosquash a9c145c, which has already 1) moved your commit to the correct position and 2) changed the action from pick to fixup:

git rebase -i --autosquash a9c145c
p 226733c first commit <-- commit with typo
f 4da3895 fixup! first commit <-- reordered commit
p 316e87f add bar

This handy trick might only save you one or two minutes, but free time compounds as well. 😉

The Craftsman Approach to Tool Selection 🪚

Earlier this year, I re-read Cal Newport’s Deep Work. The book is full of useful ideas, but one I frequently overlooked is the craftsman approach to tool selection:

[…] Identify core factors that determine success and happiness […]. Adopt a tool only if its positive impacts on these factors substantially outweigh its negative impacts. (Deep Work by Cal Newport, page 181ff.)

Cal mentions this in the context of leaving (or reducing) social media, but in my mind, this approach can be applied to all areas of life. For instance, do you need a full-fledged solution for executing HTTP requests occasionally, or isn’t curl just enough?

The key here is that the positive impacts must “substantially outweigh” the negative ones. Consider factors like bloated UIs, new dependencies in your project, maintenance burdens, and so on.

By carefully selecting the right tools, you can streamline your workflow and focus on what truly matters.

Wrapping Up

Thank you for reading this week's Weekly Tech Tidbits. I covered the importance of mentoring and how it compounds knowledge within a team, a practical tip to save time during git rebasing with fixup and autosquash, and the craftsman approach to tool selection.

I'd love to hear your thoughts, questions, and feedback in the comments below. 💬 Don't forget to subscribe for more insights 🔔 and share this post with your network! 📢

Leave a comment

Stay tuned for next week's post.