EkoPen

Our Concept
Our 3D model

🧠 Inspiration

We began with a simple observation: while AI tools are getting better at helping people write, they’re still stuck behind screens. In classrooms, notebooks, and on paper, there’s no real-time AI support for handwritten thoughts.

We were inspired by the potential of merging analog expression with modern intelligence. Whether it’s a student solving a math problem, a writer crafting a story, or someone with learning differences working through an essay, everyone deserves a supportive, intuitive writing companion that meets them where they are.

So we set out to build EkoPen — a pen that doesn’t just write; it thinks with you.

✍️ What It Does

EkoPen is an AI-powered writing assistant that provides real-time voice feedback on your handwritten content via a connected headset — blending the tactile experience of writing with the power of AI.

It recognizes what you're writing and responds in real time with:

✅ Grammar and spelling corrections
💡 Creative writing prompts
➗ Math error detection and hints
🔄 Sentence structure suggestions
🧠 Context-aware insights and questions

Instead of you prompting AI, EkoPen prompts you — sparking ideas, correcting mistakes, and helping you think more clearly on paper. You can also interact by writing questions or speaking them aloud, and the AI responds naturally through your headset.

🧠 How We Built It

For our MVP, we focused on building the software pipeline that powers the real-time AI experience:

1. Handwriting-to-Text Recognition

We trained deep learning models (CNNs + LSTMs) to interpret handwriting input and reconstruct text from stroke data. For now, this is simulated through drawn input or stylus/mouse tracking via a web app.

2. AI-Powered Feedback Engine

Once text is recognized, we run it through a lightweight natural language pipeline (based on GPT-style models) to generate grammar corrections, writing suggestions, math hints, and creative prompts.

3. Real-Time Voice Assistant

We integrated a fast TTS (text-to-speech) system that converts AI responses into spoken feedback, streamed instantly to a Bluetooth headset. Latency was optimized to stay under 200ms for natural interaction.

🧩 Challenges We Ran Into

Real-time voice feedback needed careful tuning to avoid overwhelming users — we had to strike a balance between helpful and distracting.
Latency optimization across the AI → TTS → audio pipeline required testing different hosting solutions and lightweight models.
User experience with audio-first AI was non-traditional — we spent time refining tone, frequency, and timing of voice prompts to make it feel natural.
Simulating handwriting input in the absence of hardware was tricky — we had to fake realistic pen stroke data for model training and testing.

🏆 Accomplishments We're Proud Of

Built a fully functional MVP that recognizes handwriting and delivers real-time voice feedback.
Designed and deployed a working AI feedback engine tuned for grammar, creativity, and math.
Created a smooth, non-intrusive voice agent experience that feels helpful — not robotic.
Got positive feedback from early testers, including students and teachers, on usefulness and potential.
Developed a unique interface where AI becomes your writing companion — not just your editor.

📚 What We Learned

AI is most helpful when it nudges, not lectures — tone and brevity matter.
Real-time education tools must feel instant, intuitive, and non-disruptive to build trust.
Building assistive tech requires deep empathy — especially for diverse learners with unique needs.
Start with software first — hardware can follow once the core experience is validated.

🚀 What's Next for EkoPen

Now that our software MVP is complete, we're shifting toward hardware development to bring the full vision to life:

🔩 Sensor Fusion Research: Combine IMU + optical or pressure sensors for accurate stroke capture
🧠 Custom PCB Design: Develop a compact, low-power board to handle data capture and Bluetooth communication
🔋 Power & Housing: Design a rechargeable battery system and ergonomic pen enclosure
🎙️ Headset Integration: Refine Bluetooth headset sync for seamless, low-latency voice feedback
🏫 Pilot Testing: Launch early trials in schools and learning centers
📣 Community & Content: Grow awareness through demos, storytelling, and early access campaigns