Inspiration
Not every student has equal access to education. There are people out there with phones, but no access to a good education that provides live feedback as they work. This is where teech.ai fills the gap. We aimed to create an AI-powered math teacher that's available 24/7, patient, and accessible to anyone with a smartphone, thereby bridging the education gap with technology. This isn't just for people in need, but for everyone who seeks additional help and understanding of math without jumping to the final answer immediately. One of our members has a brother who struggled to understand math from a young age because of how it was taught to him. Likewise, many students struggle with math because it can be confusing at first glance. On the other hand, there are students who do their homework late at night and want to understand how the math works without reading textbooks on proofs of how the concepts work, only because it's just confusing due to how convoluted it can be sometimes. Many often struggle without a tutor, parent, or teacher available to help.
What it does
teech.ai is a mobile AI math tutor that uses your phone's camera to see and understand handwritten math problems in real-time. Instead of just giving answers, it acts as a tutor -- guiding students through problems with hints and questions after every chat, ensuring they progress through the problem with accuracy. The app tracks the time students spend working on problems and their subproblems, adjusting its teaching approach to offer more support when students are frustrated and celebrating their progress. It also features TTS for audio guidance, particularly for individuals with impairments (e.g., dyslexia, ADHD).
How we built it
We created a rough draft of the UI/UX and interactive interface using Claude as the driver to expedite the frontend website development process. The draft was tested on an iPhone through a mobile browser, such as Safari and Chrome. The phases of the draft go as follows: the ability to check handwriting using optical character recognition (OCR) with Tesseract, the ability to check any type of handwriting and process it to give a response to help the user using Gemini 2.0 Flash, incorporate a JSON chat history for every lesson, and give the AI a voice with Fish. All of this was then polished through Xcode via Swift and became a developing iOS application ready for beta testing.
Challenges we ran into
We spent the majority of our time trying to avoid relying on an agentic AI to streamline the handwriting and text detection process. To achieve this, we used an optical character recognition library named Tesseract. However, it often detected so many artifacts that it would spew out random characters at times, which did not look pleasing. On top of that, we tried making a live handwriting and text recognition program, but it resulted in an application that consumes too much resources, which told us to take a step back and realized when the user is writing they don't want to hold the phone, so we went with letting the user be able to take snapshots as they are working to gain the encouraging feedback while doing the problem and not just after the problem.
Accomplishments that we're proud of
We were able to test different approaches, such as the OCR mentioned, and test live tracking, adapting multiple times with the user in mind to create an app that made education more accessible with 24/7 availability and voice support. Not only that, but we also worked together to streamline the process of drafting and sending parts of ideas via GitHub, thereby creating a seamless final working product.
What we learned
The concept of AI was a prominent feature in this project. Claude helped us streamline the project setup process and saved us countless hours of debugging by helping us leap over boundaries the moment we encountered a problem. Gemini helped identify and process handwritten text, as well as interact with the user. Fish is complex enough to have emotion when turning text into speech via TTS. Without the aid of Claude, Gemini, or Fish, we wouldn't be able to finish this project under the time constraints of this hackathon!
What's next for teech.ai
Our current iteration of teech.ai is relatively simple, but with more time, it can teach multiple subjects, not just math. It would make learning more feasible for everyone by adapting to the user's learning situation more quickly. The application provides structured lessons at your fingertips, offering encouraging feedback as you work on the problems. Curated for everyone, these are just a few things that can make teech.ai big everywhere!
Log in or sign up for Devpost to join the conversation.