Overstand

Inspiration

We were behind on lectures for a math topology class and needed to catch up. We were reading textbooks late at night, with nobody else to talk to, utterly confused at the mess of symbols before our eyes. We thought: what if an AI could be there to solve any confusion we had? What if we could highlight any region of the textbook, speak out a question, and have an AI respond to us as if our TA was there to guide us? Even more, what if we could add a forum for human collaboration and discussion, so everybody can share their learnings and ideas and help each other?

What it does

Overstand is a PDF library, a PDF reader, an AI educational assistant, and an educational forum all in one. We boast a feature to read through PDFs and highlight any section of text to ask the AI on for textbook-specific, context-driven answers. Not only that, but we can also speak our prompt to the AI. This means we can take pictures of complex math problems and narrate our confusion without even needing to type a letter. The AI then responds in a comforting, guiding voice, and accompanying it is a human forum for anybody to contribute their bright ideas, with a liking system to display the most relevant thoughts.

How we built it

We the frontend using React.js and Next.js. We built a 2-part backend:

Open AI, AWS Bedrock, and Webspeech services for AI context-driven answers and text-to-speech conversion.
Google firebase for server storage and data security of comments and files.

Challenges we ran into

The challenges that we faced were several fold. The usage of the three different AI apis were difficult to manage, with each of them requiring a different format and set up. Building the PDF renderer and adding the functionality of taking screenshots was also another hurdle, since there weren’t many good libraries available, those were mainly needed to be implemented natively. It brought the technical issues we needed to tackle to a whole another level.

Accomplishments that we're proud of

Some of the accomplishments we are most proud of is the fact that we integrated three AI services(text to speech, speech to text, and Large Language Models) into our project although none of us had prior experience with the technologies. We are also proud to natively implement the pdf renderer and screenshots because of the lack of available libraries. Our team built out an entire backend which is scalable and can provide a high quality experience to a wide range of users.

What we learned

Collaboration is HARD. Real-time collaboration is REALLY HARD. Leveraging the right technologies and making sure that all client states were synced was a challenge, but deeply fulfilling.
Planning is crucial — with proper planning on our stack, necessary algorithms / approaches to problems like screenshotting (which seems easy!), and even design, we saved hours of time that we would’ve otherwise been scratching our heads with.
Web technologies are versatile, but also have their limitations.

What's next for Overstand

Our first priority is making the UI more accessible by increasing the text size and pdf quality. We also plan on integrating Youtube videos and lectures into our service so that the user can ask questions about certain segments of the video. This means expanding beyond just PDFs and textbooks to many other forms of lectures and educational mediums. Further, we plan on monetizing our product with a monthly subscription.