Our project, Dyslexibility, enables users to read real-world text hands-free. Users wear their phone using a simple forehead-mounted frame. The system runs entirely through their browser, accessing the camera and microphone.
Users say: “Hey Tato, read” and the tool extracts the visible text and reads it aloud using natural speech.
To focus on a specific section of text, the user forms a rectangle using their fingers (like taking a photo), and our vision pipeline detects fingertip landmarks, crops the area between them, and only reads that text aloud.
How we built it
Finger detection using MediaPipe Hands Custom algorithm to detect when two “L-shapes” align into a rectangular bracket
OCR using Tesseract.js converted to WebAssembly for real-time performance Speech & Reading Layer Adjusted emphasis timing to mimic guided reading pacing Optional slower reading mode
Hardware: We also designed a 3D-printed forehead mount that holds the phone at an ergonomic angle. (We prototyped several versions before stabilizing the design.)
Built With
- css
- html5
- javascript
- mediapipe
- tesseract
- vite
Log in or sign up for Devpost to join the conversation.