FlowState
Inspiration
You know the moment.
Twenty tabs open. Slack notifications stacking up. Three conversations waiting for replies.
You read the same line of code five times and still haven’t written anything.
Not because the problem is hard.
Because your brain is overloaded.
The worst part is not being overwhelmed.
It is knowing you are overwhelmed and still being unable to reset.
Most productivity tools make this worse. They add dashboards, reminders, notifications, and more things to manage.
We wanted the opposite.
Instead of another tool that tracks tasks, we asked a different question:
What if software could detect when your brain is overwhelmed and step in before productivity collapses?
FlowState was built from that idea.
What it does
FlowState is an always-on desktop companion that detects cognitive overload in real time and helps users recover focus.
Instead of managing tasks, FlowState monitors behavioral and biometric signals to estimate mental state.
It builds a live Overwhelm Confidence Score using signals such as:
- blink rate
- gaze fixation and eye wandering
- breathing depth
- posture shifts
- typing speed changes
- idle time
- cursor movement patterns
When FlowState detects that the user is stuck, fatigued, or mentally scattered, it triggers targeted interventions designed to reduce cognitive friction.
Examples include:
Park & Break
FlowState captures the user’s screen and generates a short structured summary of what they were working on so they can step away without losing context.
Brain Dump
Users can talk out loud for 60 seconds. The system converts speech into structured tasks and suggests a single clear next action.
Communication Shield
If the system detects a stressed or reactive message before it is sent, it suggests a calmer rewrite.
The goal is simple:
intervene at the moment cognitive overload starts instead of after productivity is already lost.
How we built it
FlowState runs as a native Electron desktop application with a transparent always-on-top overlay.
The detection pipeline runs in a hidden renderer process using:
- MediaPipe Face Landmarker
- MediaPipe Pose Landmarker
These models extract 25+ biometric signals per second, including:
- blink detection
- PERCLOS fatigue metrics
- gaze wander ratio
- head pose estimation
- shoulder posture changes
- breathing rate estimation from chest movement
- micro-expression scoring
All signals feed into a weighted scoring model that produces a real-time engagement score.
To prevent rapid state switching, we implemented a stabilized state machine using XState which requires sustained signals before transitions occur.
For AI features we integrated:
- Gemini 2.5 Flash for screen understanding, task summarization, and tone analysis
- Deepgram Nova-3 for speech-to-text processing
- Aura-2 for AI voice feedback and coaching
The system is structured as a type-safe monorepo architecture with strict IPC contracts between Electron processes and context isolation for security.
We also implemented a double finger snap detector using FFT-based audio analysis to trigger focus music in Spotify.
Challenges we ran into
False positives were the hardest problem.
Early versions flagged normal behavior as distraction.
Coffee sips triggered fatigue alerts.
Thinking pauses triggered “stuck” states.
We solved this by introducing:
- temporal smoothing
- sustained signal thresholds
- adaptive weighting of signals
- user feedback loops that adjust sensitivity
Audio snap detection was surprisingly complex.
Distinguishing a finger snap from keyboard taps, coughs, and background noise required layered filtering:
- amplitude spike detection
- frequency band filtering (2–8 kHz dominance)
- quiet-before-spike validation
- strict double-snap timing windows
- clap rejection logic
Breathing detection from laptop webcams was fragile.
Estimating breathing rate from shoulder motion required heavy smoothing and per-user calibration due to lighting changes and camera angles.
Designing a helpful overlay without being intrusive
Because the interface is always visible, we spent significant time tuning transparency, blur, opacity, and intervention timing to avoid distracting the user.
Accomplishments that we're proud of
- Built a real-time biometric cognitive state detection system running locally on consumer hardware
- Successfully extracted 25+ behavioral signals per second using MediaPipe models
- Designed a stable engagement scoring system with a state machine architecture to prevent noisy state transitions
- Integrated multimodal AI systems for screen understanding, speech processing, and voice feedback
- Implemented a reliable audio-based double snap detection system for hands-free interaction
- Created an interface that remains visible yet minimally intrusive during normal work
What we learned
The biggest lesson was that reducing cognitive load is fundamentally different from adding productivity features.
Every feature had to pass a single test:
Does this make the user think less, or more?
If it increased cognitive effort, we removed it.
We also learned that biometric systems require personal calibration. Human behavior varies too widely for universal thresholds.
Personalized baselines are essential for the system to feel helpful rather than intrusive.
What's next for FlowState
The next phase focuses on expanding the system beyond individual signals toward personalized cognitive models.
Future development includes:
- adaptive learning of personal focus patterns over time
- cross-app context awareness (IDE, browser, communication tools)
- long-term cognitive load analytics and recovery insights
- improved physiological sensing models for fatigue detection
- team-safe productivity modes that protect deep work during collaboration hours
Our long-term vision is to build a system that acts as a cognitive safety layer for modern knowledge work, helping people stay focused without constantly managing their own productivity.
Log in or sign up for Devpost to join the conversation.