DEV Community: MABD

Building a Vim-Powered Jira Client with Compose Multiplatform & Claude

MABD — Mon, 23 Feb 2026 05:33:00 +0000

Jira is powerful — but painfully slow for keyboard-driven workflows.

As a daily vim user I am not satisfied with the experience. After years of trying to embrace it, I finally decided to build my own solution. So I built a keyboard-first Jira client powered by a custom Vim engine using Compose Multiplatform.

To be clear: this is not Jira replacement. You can still use Jira normally. I will be fetching Jira data through their official API, and show to you in a nicer way + vim

The Problem

UI changes very frequently
UI is clunky and slow
You see too many elements at your face, even if you don’t use them
Planning is hard
Mouse-heavy workflow

The Goals

Have a keyboard-first navigation system with the option to also use a mouse and keyboard. Data would be fetched from official Jira api and display on a multiplatform application on your desktop or mobile phone.

Since the data are the same, this would allow me to use this app or Jira whenever I want to. Of course while building this app, I would still be using Jira for features that won’t be supported in the app yet.

The app UI will be modern, configurable, supports small screens (phone) and large screens (tablets + laptops). This would be powered by compose multiplatform. More on that later

The architecture

Before starting with UI stuff, I need to have vim like engine to be working, at least the basic usage for now and will improve as I go. I need a way to hit keystrokes on my keyboard and convert those to events. Like move-up, move-down, click, filter, assign task, etc… This means I need to listen to keyboard strokes and handle them properly

To understand how to do this nicely, We need to know how vim is done and how it is used on the terminal

At a high level, the system is composed of:

VimEngine (mode-aware input processor)
Mode Parsers (Normal, Command)
MVI ViewModel layer
Remote + Local data sources
Compose Multiplatform UI

Vim Modes

Vim has many modes like:

normal: while navigating a file
insert: while writing to a file
command: when running commands in a file
visual: selecting text in a file
v-line: selecting lines in a file and a few more modes

For now, I will only handle normal, command modes
Each mode handles keystrokes differently

Normal Mode Parser

This mode need to parse and optionally handle keystrokes as soon as they are received. For example, if user press j, this should be parsed and understood and move down, k for moving up and so on

This is simple, we could do something like this

fun handle(c: Char): VimAction? {
    return when (c) {
        'j' -> VimAction.MoveDown
        'k' -> VimAction.MoveUp
        else -> null
    }
}

What happens if I want to handle gg (move to top of the file), you might think i can simply add it to the when statement, like this

fun handle(c: Char): VimAction? {
    return when (c) {
        // ...
        'gg' -> VimAction.MoveToTop
        // ...
}

but our handle function only send one character at a time. So we need to cache previous strokes. Then it becomes something like this

var buffer = StringBuilder()
fun handle(c: Char): VimAction? {
    when (c) {
        'j' -> {
            buffer.clear()
            return VimAction.MoveDown
        }
        'k' -> {    
            buffer.clear()
            return VimAction.MoveUp
        }
    }
    buffer.append(c)
    when (buffer.toString()) {
        'gg' -> {
            buffer.clear()
            return VimAction.MoveToTop
        }
    }
    return null
}

There are more complicated cases, like if user hit g by mistake and then he want to hit j.
This is step by step on what would happen

buffer = ""
user clicked: g
no match -> buffer = "g"
user clicked j
no match -> buffer = "gj"

Now at this point no matter what user hits, buffer would keep increasing. We can clear buffer if user click esc but is not the vim way to handle it.

Partial Pattern Matching

If you try this in vim, hitting g then j would move up. How so?
This is called partial matches. gj is not a valid keybinding, but j itself is, so g is ignored and j will run

If no exact match found, we try last n-1 characters on the buffer (n = number of characters in buffer), if that also had no match we try n-2 and so until no more characters in the buffer. But if we found a partial match, we return the corresponding vim action and we clear the buffer

This type of pattern matching is needed in normal mode parser.

Command Mode Parser

Here we need to have a predefined grammar on our commands. Let’s start with a simple and familiar one

[verb] [target] (args)

for example:
:status done (change task status to done)
:assign bob (assign task to bob)

In this mode, we don’t want to parse on each keystroke, instead we keep adding to buffer and wait for user to hit enter to apply the command or esc to cancel it

We should have a predefined vocabulary on each verb, then verb to target maps or something to know what we can handle and what we can’t

Using power of sealed interface we can do this nicely as follows

sealed interface CommandVerb {  

    fun toVimAction(target: String?, args: List<Arg>): VimAction?  

    data object Status: CommandVerb {  

        override fun toVimAction(target: String?, args: List<Arg>): VimAction? {  
            if (target.isNullOrBlank()) return null  
            val taskStatus = TaskStatus.getFrom(target) ?: return VimAction.Error("Unknown status=$target")  
            return VimAction.MoveTaskTo(taskStatus)  
        }  
    }  

    data object Assign: CommandVerb {  
        override fun toVimAction(target: String?, args: List<Arg>): VimAction? {  
            if (target.isNullOrBlank()) return null  
            return VimAction.AssignTo(target.lowercase())  
        }  
    }  

    data object Help: CommandVerb {  
        override fun toVimAction(target: String?, args: List<Arg>): VimAction {  
            return VimAction.ShowHelp  
        }  
    }

    companion object Companion {  
        fun getFrom(verbName: String): CommandVerb? {  
            return when (verbName.lowercase()) {  
                "status" -> Status  
                "assign" -> Assign  
                "help" -> Help  
                else -> null  
            }  
        }  
    }
}

Using regex we parse the command, and from it we get Command Verb like so

val verb = getVerbFromCmd(cmd)
 CommandVerb.getFrom(verb)?.toVimAction(target, args)

We need to care about edge cases like:

What happens if verb is invalid
What happens if target is invalid and so on

Creating The Engine

Since our mode parsers are ready now we can start develop the engine

Our engine needs NormalModeParser and CommandModeParser as params, the engine also need to know the current vim mode to be able to know where to send keystrokes

class VimEngine(  
    private val normalModeParser: ModeParser,  
    private val commandModeParser: ModeParser,  
) {  
    val mode: VimMode = // something

    private val modeToParser = mapOf(  
        VimMode.Normal to normalModeParser,  
        VimMode.Command to commandModeParser,  
    )  

    fun handleKey(key: VimKey) {  
        val parser = modeToParser[mode.value]  
        val vimAction = parser?.parse(key)  

        // handle vimAction
    } 
}

But wait, how should we change vim mode? and how is responsible for that.

What makes most sense is that the engine is the one holding vimMode and exposing that as a flow. I decided to make the parser return vim mode change, this way VimAction would only represent an action to be made later by ui and vim mode change is only meant for the VimEngine to see

so, I updated mode parser function to return ParserReult


data class ParseResult(  
    val action: VimAction? = null,  
    val nextMode: VimMode? = null  
)

fun handle(c: Char): ParseResult {
// ...

then handleKey function in VimEngine becomes as follows

fun handleKey(key: VimKey) {  
    val parser = modeToParser[mode.value]  
    val parseResult = parser?.parse(key)  

    parseResult?.nextMode?.let { nextMode ->  
        scope.launch { _mode.emit(nextMode) }  
    }  

    parseResult?.action?.let { action ->  
        scope.launch { emit(action) }  
    }
}

What I have currently:

User hit keys on keyboard → they get handled by NormalModeParser
When user hit : NormalModeParser assign nextMode=VimMode.Command in ParseResult so mode switches
Next keystrokes will be handled by CommandModeParser

So far so good.

Later I can add

more parsers,
more keybindings to normal mode
and more command to command mode easily

        ┌─────────────┐
        │  Vim Engine │
        └──────┬──────┘
               │
        ┌──────┴──────────┐
        ▼                 ▼
┌──────────────┐  ┌──────────────┐
│    Normal    │  │   Command    │
│     Mode     │  │     Mode     │
│    Parser    │  │    Parser    │
└──────────────┘  └──────────────┘

Hook To UI

Once the engine was stable, the next challenge was integrating it cleanly with the Compose UI layer.

I am using compose and handling keystrokes is straightforward. I listen to onKeyEventModifier, get the key, and sent it to ViewModel to be sent later to VimEngine.

I had a screen with a vertical list of tasks and I was navigating through them in vim keybinding, switching between modes, etc.. all is working

But then I wanted to show a task details, side-by-side with task list screen. The standard these days in compose is using multi-pane view. On the left i have tasks list, and on the right I have task details view. I did this and it looked nice.

┌─────────────────────────────────────────────────┐
│                                                 │
│   ┌───────────────┐   ┌───────────────────┐     │
│   │               │   │                   │     │
│   │               │   │                   │     │
│   │  tasks list   │   │   task details    │     │
│   │               │   │                   │     │
│   │               │   │                   │     │
│   └───────────────┘   └───────────────────┘     │
│                                                 │
└─────────────────────────────────────────────────┘

Why I Chose Multiple Vim Engines per Pane

When I introduced the multi-pane layout (tasks list + task details), an interesting architectural question appeared:

Who owns the keyboard behavior when multiple panes are visible?

Each pane had very different interaction semantics:

Task list → navigation heavy (j, k, gg, filtering…)
Task details → editing, actions, different commands
Future panes → unknown behaviors

I considered three approaches.

Option 1 — Dynamic Keymap Switching

Switch keybindings whenever focus changes.

Pros:

Single engine instant
Simple mental model initially Cons:
Keymap mutation at runtime
Harder to reason about state
Become fragile as number of panes grows

This felt convenient short-term but risky long-term

Option 2 — Swap Parsers Inside One Engine

Keep one engine but replace it’s parsers based on focused pane.

Pros:

Still one engine
Some separation of behavior (different parsers for different focus panes) Cons:
Engine becomes focus-aware. This prevent it of being re-used in another projects
Parser lifecycle becomes harder to track
Increased coupling between UI and engine

This improved separation slightly but still mixed responsibilities.

Option 3 — Multiple Vim Engines (Chosen)

Each focusable pane owns its own VimEngine instance.

The ViewModel simply routes keystrokes to currently focused pane’s engine

Pros:

Strong isolation between panes
Each pane can evolve independently
No runtime mutations of keymaps
Simpler mental model per engine
Future-proof for more panes
Enable parsers sharing when desired Cons:
More engine instances in memory
Slightly more wiring in ViewModel

For this applications the tradeoff was clearly worth it.

The Key Design Principle

The decision was guided by one rule:

Keyboard behavior is contextual UI state, not global application state.

By giving each pane its own engine:

focus becomes the only routing concern
engines remain pure and predictable
adding new panes does not increase complexity of existing ones

In practice, this made the system much easier to extend than the single-engine approaches.

Why This Matters for Future Growth

This design unlocks several things almost for free:

Different Vim capabilities per pane
Experimental keymaps in isolated areas
Plugin-like future architecture
Potential extraction of the Vim engine as a reusable library

Most importantly, it keeps the architecture honest: each UI surface owns its own interaction model.

Small But Important Optimization

Even though I use multiple engines, parsers themselves can still be shared when behavior overlaps. This avoids unnecessary duplication while preserving isolation where it matters.

                    Key Press
                        │
                        ▼
                ┌────────────────┐
                │   ViewModel    │
                │ (focus aware)  │
                └──────┬─────────┘
                       │
          ┌────────────┴────────────┐
          │                         │
          ▼                         ▼
 ┌─────────────────┐      ┌─────────────────┐
 │ Tasks List Pane │      │ Task Details    │
 │   VimEngine     │      │   VimEngine     │
 └────────┬────────┘      └────────┬────────┘
          │                         │
          ▼                         ▼
   Normal / Command          Normal / Command
        Parsers                   Parsers

The complete flow looks like this

User presses key
        │
        ▼
┌──────────────────────┐
│      Compose UI      │
│   onKeyEvent(...)    │
└──────────┬───────────┘
           │
           ▼
┌──────────────────────┐
│      ViewModel       │
│ (focus-aware router) │
└──────────┬───────────┘
           │ routes by focus
           ▼
┌──────────────────────┐
│      VimEngine       │
│  (mode-aware parse)  │
└──────────┬───────────┘
           │ emits
           ▼
┌──────────────────────┐
│      VimAction       │
└──────────┬───────────┘
           │ mapped to
           ▼
┌──────────────────────┐
│     ScreenIntent     │
└──────────┬───────────┘
           │ handled by
           ▼
┌──────────────────────┐
│   ScreenInteractor   │
│ (business logic)     │
└──────────┬───────────┘
           │ produces
           ▼
┌──────────────────────┐
│   Reducer (MVI)      │
│   uses currentState  │
└──────────┬───────────┘
           │ emits
           ▼
┌──────────────────────┐
│      New State       │
└──────────┬───────────┘
           │
           ▼
        Compose UI
          recomposes

UI

Switching Focus

To be able for the viewModel to know which pane is focused I created a state for it and saved it in viewModel. Then based on emitted vim actions I would know which pane is focused

For example:

when user click enter on a task in tasks list pane → switch focus to task details pane
when user click esc on task details pane → switch focus back to tasks list pane
I even went step further and added vim like keybinding for this in NormalModeParser
- ctrl-l: switch focus to pane on the right (in this case task details)
- ctrl-h switch focus to pane on the left (in this case tasks list)

Of course using mouse clicks here would also work and switch focus properly

Reminder: this is not a vim-only app, but vim based so mouse still works as expected (screen touches as well for mobile devices)

UI Features

Tasks List: auto scroll when user hit j, k, gg, G
Animated task status update, with loading progress while it’s getting updated
Stacked Notification system: show info, error, warning notifications at the top-right corner of the app, with auto-disappearing after 3 seconds
Highlight focused pane
Popup to show all available keybindings and what each do

More Vim Features

Here is a list of vim feature I also supported

Repeatable actions Repeating last command by clicking on .
In Command Mode: click arrow-up/arrow-down it would show previous/forward commands executed
Each parser has it’s own buffer
Vim Engine: Expose buffer for the currently working parser, show buffer content on UI
NormalModeParser:
- Created default keybinding
- Extra keybinding: to support configurable keybindings later

Data Layer

This app is intended to be used offline. So I need a local data source and remote data source (Jira). They both expose my domain level models.

Remote Data Source (interface)
- Real Implementation: abstract way Jira models, and only return back my domain level models
- Fake Implementation for testing
Local Data Source (interface)
- In-Memory Implementation: store and cache data to memory
- Db Implementation (to be done) save into DB. This is needed for offline mode support

Claude Code

It’s 2026, not using AI in the development workflow would be a missed opportunity.

I used LLM tooling (Claude Code) strategically to accelerate implementation while keeping full architectural ownership and code review responsibility.

My rule was simple:

AI can generate — but I design, verify, and own the system.

What AI helped the most

UI Scaffolding

Since the early focus of the project was the interaction model rather than visual polish, I used Claude to scaffold several UI components.

With well-scoped prompts and clear context, most components were generated correctly in one pass. This allowed me to:

move faster in early iterations
avoid spending time on repetitive Compose boilerplate
keep focus on the Vim engine and state flow

As the product matures, I expect to take more manual control over UX refinement.

API Layer

The Jira integration layer is something I’ve implemented many times professionally, so it was a good candidate for delegation.

I provided Claude with:

Jira API documentation
my project structure conventions
interface contracts (real + fake implementations)
error-handling expectations
domain model mappings

Because the constraints were explicit, the generated code was:

Clean
Testable
Idiomatic
and required only light review adjustments

This is exactly the kind of work where AI currently provides the most leverage.

Unit Testing

The Vim engine and parsers have many edge cases, and comprehensive unit testing is essential but time-consuming.

My workflow was:

I defined the test scenarios
Claude generated the test implementations
I reviewed and refined them
GitHub Actions enforce them on every PR

This gave me broad test coverage quickly while maintaining confidence in correctness.

What AI Did Not Own

The following remained fully manual:

overall architecture
Vim engine design
mode system
state flow (MVI)
focus routing model
concurrency decisions

AI accelerated the build — but the system design decisions remained human-driven.

Takeaway

Used carelessly, AI can produce fragile systems.

Used deliberately, it becomes a powerful force multiplier.

In this project, the goal was never to replace engineering judgment — only to remove unnecessary friction from the implementation process.

Things To Improve

This is still the first version of the app. The core interaction model is working well, but several areas need to mature before this could be considered production-ready.

High priority

Proper Jira API authentication

Currently the app uses a simple API token. Supporting OAuth and improving token handling will be required for real-world usage.
Offline mode support

The data layer is designed for it, but the database implementation and sync strategy still need to be completed.
Smarter cache invalidation

Right now caching is basic. As usage grows, I will need more deliberate invalidation and refresh strategies to avoid stale task data.

Medium priority

Extract VimEngine as a reusable library

The engine is already mostly decoupled. With some cleanup it could become a standalone module usable in other projects.
More Vim motions and text objects

The current implementation focuses on navigation and commands. Expanding motion coverage will improve muscle-memory compatibility for heavy Vim users.

Longer-term explorations

Performance tuning under heavy key input

As the number of panes and commands grows, I want to measure and optimize keystroke latency and buffering behavior.
Plugin-style extensibility

The multi-engine design opens the door for pane-specific extensions. I’m interested in exploring how far this model can scale.

Final thoughts

What started as a small experiment has quietly become part of my daily workflow. I now use it daily at work on daily standup (I am the scrum master, I can do that 😁)

Simple operations — like filtering tasks or moving an issue from todo to done are now muscle memory. For example, md (move to done) is often faster than reaching for the mouse and navigating multiple menus.

Interestingly, my teammates initially assumed I was using some existing tool rather than something custom-built. That reaction alone was a strong signal that the interaction model is heading in the right direction.

There is still plenty of work ahead, but the core bet is already paying off: when keyboard interaction is treated as a first-class architectural concern, the entire experience changes.

The goal was never to replace Jira, only to make working with it finally feel fast.

I Built a Tool to Track My Open Source Contributions

MABD — Mon, 22 Dec 2025 06:41:06 +0000

Github contributions graph is great at showing activity, but it does not answer the question: what open source projects I have contributed to.

I wanted to display open source projects I have contributed to on my personal website. I can do this manually. However, this can get annoying and add extra thing to remember.
I want to show projects I contributed to, how many PR’s merged, lines of code contributed. Github does not surface this easily, so i built a tool to do so.

The Problem

If you contribute to external repositories (projects you don’t own) Github buries this info. You can get it manually by searching your PR’s, but their is no API endpoint that says “give me all this user’s contributions to external repositories”

I wanted:

List of external projects contributed to
Number of merged PR’s per project
Commit count and number of lines added/removed (per project)
JSON output I can feed to my website

So I created gh-oss-stats

The Approach

The core insight is to use Github’s search query

author:USERNAME type:pr is:merged -user:USERNAME

This find all pull requets:

Authored by you (author:USERNAME)
That are PR’s not issues (type:pr)
That are merged (is:merged)
For repos you don’t own (-user:USERNAME) That's your OSS contribution history in one query.

Request looks like this:

https://api.github.com/search/issues?q=author:mabd-dev+type:pr+is:merged+-user:mabd-dev

output looks like this:

{
  "total_count": 20,
  "incomplete_results": false,
  "items": [
      {
          {
      "url": "https://api.github.com/repos/qamarelsafadi/JetpackComposeTracker/issues/9",
      "repository_url": "https://api.github.com/repos/qamarelsafadi/JetpackComposeTracker",
      "labels_url": "https://api.github.com/repos/qamarelsafadi/JetpackComposeTracker/issues/9/labels{/name}",
      "comments_url": "https://api.github.com/repos/qamarelsafadi/JetpackComposeTracker/issues/9/comments",
      "events_url": "https://api.github.com/repos/qamarelsafadi/JetpackComposeTracker/issues/9/events",
      "html_url": "https://github.com/qamarelsafadi/JetpackComposeTracker/pull/9",
      "id": 3204496021,
      "node_id": "PR_kwDONQBujs6diLmP",
      "number": 9,
      "title": "🔧 Refactor: Add Global Theme Support for UI Customization",
      "user": {...},
      "labels": [...],
      "state": "closed",
      },
      ...
  ]
}

From there, it's a matter of:

Fetching PR details (commits, additions, deletions)
Enriching with repo metadata (stars, description)
Aggregating into useful statistics

Architecture Decision: Library First

I built this as a Go library with a CLI wrapper, not just a CLI tool. The core logic lives in an importable package:

import "github.com/mabd-dev/gh-oss-stats/pkg/ossstats"

client := ossstats.New(
    ossstats.WithToken(os.Getenv("GITHUB_TOKEN")),
    ossstats.WithLOC(true), LOC: lines of code
)

stats, err := client.GetContributions(ctx, "mabd-dev")

This means I can use the same code in:

The CLI tool (for local use)
GitHub Actions (automated updates)
A future badge service (SVG generation)
Anywhere else I need this data The CLI is just a thin wrapper that parses flags and calls the library.

Handling GitHub's Rate Limits

GitHub's API has limits: 5,000 requests/hour for authenticated users, but only 60 requests/hour for the Search API. For someone with many contributions, you can burn through this quickly.

The tool implements:

Exponential backoff on rate limit errors
2-second delays between search API calls
Controlled concurrency (5 parallel requests for PR details)
Partial results if rate limited mid-fetch

The Output

Running the tool produces JSON like this:

{
  "username": "mabd-dev",
  "generatedAt": "2025-12-21T06:46:57.823990311Z",
  "summary": {
    "totalProjects": 7,
    "totalPRsMerged": 17,
    "totalCommits": 58,
    "totalAdditions": 1270,
    "totalDeletions": 594
  },
  "contributions": [
    {
      "repo": "qamarelsafadi/JetpackComposeTracker",
      "owner": "qamarelsafadi",
      "repoName": "JetpackComposeTracker",
      "description": "This is a tool to track you recomposition state in real-time !",
      "repoURL": "https://github.com/qamarelsafadi/JetpackComposeTracker",
      "stars": 94,
      "prsMerged": 2,
      "commits": 14,
      "additions": 181,
      "deletions": 78,
      "firstContribution": "2025-06-14T20:55:24Z",
      "lastContribution": "2025-07-21T21:39:53Z"
    },
    ...
  ]
}

This feeds directly into my website's contributions section.

Using It

Installation

go install github.com/mabd-dev/gh-oss-stats/cmd/gh-oss-stats@latest

Basic Usage

# Set your GitHub token
export GITHUB_TOKEN=ghp_xxxxxxxxxxxx

# Run it
gh-oss-stats --user YOUR_USERNAME

# Save to file
gh-oss-stats --user YOUR_USERNAME -o contributions.json

Automating with GitHub Actions

I run this weekly via GitHub Actions to keep my website updated automatically:

name: Update OSS Contributions

on:
  schedule:
    - cron: '0 0 * * 0'   # Weekly on Sunday
  workflow_dispatch:      # Manual trigger

permissions:
  contents: write

jobs:
  update-stats:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-go@v5
        with:
          go-version: '1.25'

      - name: Install gh-oss-stats
        run: go install github.com/mabd-dev/gh-oss-stats/cmd/gh-oss-stats@latest

      - name: Fetch contributions
        env:
          GITHUB_TOKEN: ${{ secrets.GH_OSS_TOKEN }}
        run: |
          gh-oss-stats \
            --user YOUR_USERNAME \
            --exclude-orgs="your-org" \
            -o data/contributions.json

      - name: Commit changes
        run: |
          git config user.name "github-actions[bot]"
          git config user.email "github-actions[bot]@users.noreply.github.com"
          git add data/contributions.json
          if ! git diff --staged --quiet; then
            git commit -m "Update OSS contributions"
            git push
          fi

Now my website always has fresh data without any manual work.

Displaying on My Website

On mabd.dev, I read the JSON file and render it. The exact implementation depends on your stack, but the data structure makes it straightforward:

Loop through contributions array
Display repo name, stars, PR count
Show totals from summary
Link to the actual repos

The JSON is the contract — however you want to display it is up to you.

What I Learned

GitHub's Search API is powerful but quirky. The -user: exclusion syntax does not exclude repos you own on your organization. I had to do custom logic to detect that.

Library-first design pays off. Building the core as an importable package meant the CLI came together in under an hour. It also means future tools (like a badge service) can reuse 100% of the logic.

What's Next

I'm planning to build a companion service gh-oss-badge that generates SVG badges you can embed in your GitHub profile README:

markdown

![OSS Stats](https://oss-badge.example.com/mabd-dev.svg)

Same data, different presentation. The library-first architecture means this service will just import gh-oss-stats/pkg/ossstats and add an HTTP layer on top.

If you want to track your own OSS contributions, give gh-oss-stats a try. It's open source (naturally), and contributions are welcome

Resources

Github api docs: https://docs.github.com/en/rest?apiVersion=2022-11-28
Github api rate limit: https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api?apiVersion=2022-11-28
Authenticating to rest api: https://docs.github.com/en/rest/authentication/authenticating-to-the-rest-api?apiVersion=2022-11-28

Search Engine from Scratch — Part 1: The Inverted Index

MABD — Sun, 14 Dec 2025 11:03:30 +0000

I started something big — building a text-based search engine from scratch 🔍

Why this?

🧠 Zero experience in search engines = massive learning opportunity
💡 We use search daily (browser, Spotlight, Windows Search) yet rarely think about how it works
⚙️ Pure algorithmic challenge — no backends, no APIs, just data structures and efficiency

The goal: a fast, pluggable search tool I can hook into other projects.

My current knowledge

As a developer, when I hear "search feature," the first thing that comes to mind is a simple algorithm:

for all texts -> find text that contains word (case insensitive)

That's it. The trivial "contains" algorithm.

This is usually good enough for a large portion of projects. But what happens when the data is huge, or the query is more than just a single word?
Hmmm, this is where it gets complicated, and from here, I had no idea what to do.

Information Retrieval

Information retrieval (IR) is finding material of an unstructured nature (usually text) that satisfies an information need from within large collections.

Let me explain with an example. Say you have a list of 100 words and their meanings. When you want to find a word's meaning, you traverse the list until you find it. But what happens when the list grows to 100,000 words? There's no way you can traverse the whole list every single time.

Those 100,000 words are unstructured and the collection is very large.

To retrieve information efficiently, we need a better approach. As a developer, you probably already know one solution: sorting.

Sort the words alphabetically and finding any word becomes trivial, even if the collection grows to 10 million entries.

Search engines apply similar thinking: take a huge collection of data, then use algorithms and data structures to organize it for fast future lookups.

Search Engine V1

Objective: choose a folder on you machine and search for any word in it

W’ll build our engine based on files, let’s call them documents

type Document struct {
    ID   int
    Path string
}

We want to search for words across a folder, so ideally our data structure would map each word to the list of files where it appears. This is called an inverted index, "inverted" because instead of mapping documents → words, we map words → documents.

Each location where a word appears is called a posting (think of it as "posting" the document to that word's list):

type Posting struct {
    DocID int
}

type Index map[string][]Posting

Now we need to process all files in our folder and extract unique words. This process is called indexing.

The algorithm is straightforward: for each file, split by whitespace, then trim leading and trailing spaces from each word.
But a problem appears — you end up with things like: (IR), Objective:, backends., }

So we need to remove symbols and punctuation to get clean words. Let's call these cleaned words tokens.

Now we can build our index:

func indexDocument(doc Document, index Index) Index {
    fileContent := getFileContent(doc.Path)
    tokens := tokenize(fileContent)

    for _, token := range tokens {
        index[token] = append(postings, Posting{DocID: doc.ID})
    }
    return index
}

After indexing all files, we're ready for queries.

Querying

For now, let's keep it simple, searching for a single word:

func getPostings(token string) []Posting {
    return index[token]
}

That's it! We get all documents where our token exists.

This will get more exciting later when we track positions within each document where the query appears. Try implementing that yourself 🙂

You can find the full code on github