Lume Zero

Lume Zero is a low-carbon, high-impact remote assistant app built on Serverpod 3 enabling remote collaboration without the impact and economic cost of video streaming.

Comment

Inspiration

In a world that uses by high-bandwidth video conferencing, regions with unstable connections, low bandwidth or high latency are often left behind. Geography or extreme weather frequently disrupt connectivity and screen sharing become unusable or a blocky pixelated mess. We wanted to build a bridge, something climate friendly, that would use less bandwidth but should not sacrifice screen clarity. By using intelligent streaming captures we believe digital collaboration can be made more inclusive and resilient.

What it does

Lume Zero is a streamlined screensharing application designed for efficiency. Instead of streaming a continous, heavy video feed, it uses FFmpeg to capture and transmit optimized screenshots via Serverpod. The "helper" user can talk with a digital butler, powered by Gemini, to analyze the shared screen in real-time to provide instant context and troubleshooting. We used a luminous effect on the cursor that allows dropping hints and trails to make following more complex workflows more understandable even at lower frame rates as a functional accessibility feature.

How we built it

The core engine is built on top of FFmpeg, which handles the heavy lifting of screen capture and image compression to try to stay within a "green" bandwidth budget. The backend is powered by Serverpd, leveraging the Dart ecosystem to create a high-performance, scalable link between users. For the AI layer, we integrated Gemini to "view" the current screen data and provide intelligent feedback as a smart assistant. On the frontend the user is guided by the helper's "Luminous" cursor trail and bright hints that can be dropped on specific parts of the screen.

Challenges we ran into

The biggest challenge was dealing with image quality and latency. The first version used WebRTC but ran into issues which freezing and low 'snapshot' sounds when attempting to take screenshots. We had to find an ideal solution to send just enough data based on the users bandwidth. In the end, we opted to allow an Eco/Balanced/High setting that would control the frequency of how often the screen was sent.

Accomplishments that we're proud of

The application is streamlined enough that there is near zero lag for users on restricted bandwidth.

What we learned

Prior to this, we had zero experience with deploying ServerPod and thought it would be a lot of having to deal with web socket management. Instead we were able to quickly use ServerPod as a signaling layer, allowing us to run a lightweight instance to ensure that the infrastructure remains as climate-friendly as the app. Reconnection logic seemed to be handled gracefully as well.

What's next for Lume Zero

Moving to Delta encoding would allow us to reduce the amount of data sent even more but was not possible in the current timeframe. Also we are exploring enhancing the Gemini Butler integration to allow enhanced answers by utilizing a pre-existing knowledge base based on the application that the user is currently using.

Built With

Share this project:

Updates