Livepair

Inspiration

This was the kind of project I had been aiming for: a pair assistant that I can activate whenever I want with just a few clicks. It is mainly a desktop application, though it can also have a web version, with a smooth overlay on top of my window. So whenever I want to ask for a suggestion, for example, I can simply open it and press “Talk.”

What I find especially compelling is that it can truly function as a pair-programming assistant. Since it is not just another web application with a chat box, but something that can stay in my window with almost no friction, it really feels like my own pair assistant.

What it does

Livepair is a real-time multimodal desktop assistant that can talk with the user, understand screen context, and show live transcripts directly in the chat UI. While the conversation happens, both user speech and assistant responses are transcribed in real time, making the interaction visible and easy to follow.

The app also includes configurable personality, voice, image quality, theme settings, and developer/debug options, giving users control over how the assistant behaves and how the experience looks.

How we built it

We built Livepair as a desktop application using Electron, with a React + TypeScript frontend for the overlay interface and NestJS for the backend API. The app connects to Gemini for real-time multimodal interactions, while the backend handles services such as session/token management and cloud-connected infrastructure.

Challenges we ran into

One of the biggest challenges was not just building the features, but finding the right development strategy to move fast without losing precision. To keep progress high, the project was often developed across three parallel worktrees, which required a strong understanding of the codebase structure and careful coordination to minimize conflicts and keep the workstreams aligned.

Another challenge was managing the overall complexity of a real-time multimodal desktop assistant, where UI, backend, streaming interactions, and stability all needed to evolve together. On the other hand, working with the Gemini API was a smoother part of the process, thanks to its strong documentation and clear developer experience.

Accomplishments that we're proud of

To get done an application with a high potential.

What we learned

This project was a strong hands-on opportunity to learn how Google Cloud, infrastructure as code, Electron desktop applications, and the Gemini APIs work together in a real product. It also taught us a lot about how to use AI to improve development workflows, move faster, and make better technical decisions during implementation.