🦊 WhatDoesDaFoxSay

Read with Confidence, Grow with Patience
A calm, playful reading companion designed for children with dyslexia.
WhatDoesDaFoxSay combines speech recognition, micro-expression analysis, and eye-tracking to create a personalized reading experience that helps children practice comprehension — while empowering parents with trust and transparency.

🌎 Social Impact

Dyslexia affects 1 in 5 children worldwide, often leading to frustration, anxiety, and loss of self-confidence. Traditional reading tools fail to adapt to how neurodivergent minds process words — especially when compounded by modern distractions like social media and cognitive overload.

WhatDoesDaFoxSay provides a space where learning feels joyful, not judgmental — helping children build patience, rhythm, and reading confidence while giving parents insight into their child’s progress and peace of mind through science-backed transparency.

🧠 Inspiration

As a team, we’ve personally experienced or witnessed the struggles of dyslexia — the pauses, the skipped lines, the feeling of falling behind despite trying harder than anyone else.
We asked ourselves:

“What if learning to read could feel more like a conversation — not a test?”

We built WhatDoesDaFoxSay to bridge empathy with technology — turning practice into play and helping children rediscover their voice.

💡 What It Does

🗣️ Speech Recognition & Reading Practice

Children read aloud while ElevenLabs listens, transcribes, and identifies tricky sections — analyzing tone, fluency, and stutters in real time.

👀 Eye Tracking (Focus Funnel)

Using advanced gaze tracking, the app identifies where a reader’s eyes linger or skip — funneling insights from paragraph → sentence → word to pinpoint comprehension gaps.

🙂 Micro-Expression Understanding

If the child appears confused, frustrated, or tired, micro-expression analysis triggers adaptive hints or encouragement — making every interaction emotionally intelligent.

🦊 Fox Mode (For Parents)

A serene space designed for parents — offering articles, scientific studies, and trustworthy stats about dyslexia, ADHD, and child learning.
It’s not a “dark mode.” It’s a peace mode.
Parents can explore the models we use, understand our accuracy, and see the research behind the app’s design.

🌟 Key Benefits

Adaptive Learning: Every child receives feedback tailored to their pace and focus level.
Emotionally Aware: Detects confusion and frustration in real time.
Parent Trust: Scientific transparency and model explainability build confidence in our platform.
Data Privacy: All video and audio processing happens securely — none of the data is sold or reused.
Inclusive Design: Warm, dyslexia-friendly fonts (Atkinson Hyperlegible), pastel farm colors, and gamified UX to reduce cognitive load.

🚀 Use Cases

Early reading intervention for children (Grades 2–4)
Home or classroom dyslexia practice companion
Reading fluency tracking for speech therapists and educators
Parental dashboards for progress tracking and emotional reassurance

🛠️ How We Built It

🖥️ Frontend

Vanilla HTML/CSS/JS for lightweight performance
Atkinson Hyperlegible font and farm-themed color palette to reduce reading strain
Auth0 for secure login via Google, Microsoft, GitHub, Facebook, Passkeys (fingerprint/eye), with ReCaptcha and password-reset via Email/SMS
FirestoreDB for storing user profiles and reading progress

⚙️ APIs & ML

ElevenLabs for expressive speech recognition and child-friendly voice synthesis
Gemini API for adaptive reading comprehension and feedback generation
OpenFace + MediaPipe Iris for micro-expression and gaze estimation
Firebase Functions for backend event tracking and analytics

🔄 Data Flow

Child reads aloud → audio + webcam input analyzed locally → Gemini summarizes focus + fluency → Firestore stores metrics → Fox Mode displays parental analytics and resources.

🚧 Challenges We Overcame

Training real-time micro-expression models to differentiate “confused” vs. “focused”
Ensuring privacy for minors — running gaze and facial inference entirely on-device
Creating an emotionally neutral design that feels safe for children yet informative for parents
Integrating Auth0’s multi-auth methods seamlessly with Firebase and ReCaptcha
Building a 3-layer funnel (paragraph → sentence → word) pipeline for focus mapping

🏆 Accomplishments

Three-Pronged Reading Engine: Real-time fusion of speech, gaze, and emotion for personalized feedback
Fox Mode: A first-of-its-kind parental space blending transparency, trust, and neuroscience-backed education
Secure Identity Pipeline: Auth0 + Firestore + ReCaptcha for verified, protected family accounts
High Accuracy Models: Achieved >90% accuracy in gaze calibration (MediaPipe + Kalman filter) and robust AU detection (OpenFace 3.0)

📚 What We Learned

Dyslexia is not a limitation — it’s a different pattern of learning that thrives under adaptive design
Parents need emotional trust as much as children need encouragement
Accessibility isn’t about simplification — it’s about personalization
Combining AI with empathy makes education genuinely transformative

🚀 Next Steps

Add voice-emotion modeling for richer feedback during reading aloud
Integrate fine-tuned stutter detection from the Efficient-Stuttering-Event-Detection model
Expand Fox Mode into a verified research portal (with direct links to academic studies and impact metrics)
Build teacher dashboards with class-wide analytics for early intervention
Deploy mobile-first version with offline reading sessions

❤️ Why WhatDoesDaFoxSay

Because no child should feel “slow” for learning differently.
Because parents deserve peace of mind, not worry.
And because the fox believes — with patience, every child can learn to read out loud and with confidence.