DeepSky

Live feedback for the pilot
"Backend" setup for demo - live simulator and a twitch stream
Analzyed frame and Radar view

Inspiration

The idea for DeepSky came from our experience with general aviation: small aircraft pilots often lack the advanced collision avoidance systems found in commercial aviation. With rising air traffic and limited visibility in busy skies, we saw an urgent need for an affordable, AI-driven solution to enhance safety and give pilots peace of mind.

What it does

DeepSky uses cutting-edge AI to detect aircraft in real-time by analyzing live video streams from onboard cameras. It identifies potential collision risks and delivers near-instant voice feedback to pilots using ElevenLabs low-latency Text-To-Speech, acting as an extra set of vigilant eyes in the cockpit. Our system empowers pilots to focus on flying while we handle the scanning.

How we built it

We developed DeepSky by integrating high-performance AI models with the simple camera hardware. The system processes video feeds through visual large language model (GPT-4o-mini), flags aircraft, and triggers voice alerts generated by lightweight TTS model (ElevenLabs flash) via a custom audio interface - all optimized to run seamlessly in real-time interactions.

Challenges we ran into

We managed to write the PoC very quickly by utilizing large AI models, which have quite substantial latency. For demo we wanted to use the most capable models, but if we would want to optimize for realtime interactions we would need to tune smaller models on some custom prompt format so we would maintain high accuracy with better performance.

Accomplishments that we're proud of

We’re thrilled to have built a working prototype that accurately detects aircraft in real-time and delivers voice alerts in under 3 seconds. Successfully testing it in simulated flight scenarios-and hearing it perform flawlessly-proved our concept. Plus, keeping costs low means it’s viable for small aircraft operators.

What we learned

We learned that balancing AI accuracy with real-time performance is a tightrope walk, but doable with clever optimization. Cockpit ergonomics matter—pilots need alerts that cut through the chaos without overwhelming them. And collaboration across AI, hardware, and aviation expertise will be key to making this work in production.

What’s next for DeepSky

Next, we’re refining the system with more flight data to boost detection accuracy across all conditions. We’re also pursuing partnerships with avionics manufacturers and regulatory bodies to certify DeepSky for real-world use. Our goal? Get this into cockpits, save lives, and scale to a must-have safety standard in small aviation.