PaperSnap

Collaborating remotely on a whiteboard, but don't own a fancy stylus pen? This product converts your hand-written equations into LaTeX format and renders them onto a live canvas!

Comment

Inspiration

Education is fundamental to the progression of society and currently, the demand for online and remote teaching resources is high. Many people may not be optimally equipped with products such as tablets and styluses so working from paper is the only viable solution. The quality of internet connection and readability of handwriting can reduce the ability to learn and teach. Papersnap provides a quick, simple, and easy solution to this problem, allowing for a better educational experience for all.

What it does

This full-stack app takes a photo of your handwritten notes and even mathematical equations before turning them into mark-up language format to render them onto a communal whiteboard on the web. Each user submission would be rendered as a resizable, draggable box on a web canvas and anyone can contribute.

How we built it

Flutter - Mobile app to take photos and send it to our server (which is connected to Azure Cognitive Services).

React and Express.js - Full-stack React web app to handle images, optical character recognition, and database with MongoDB

Python - Optical Character Recognition (OCR) for text recognition using machine learning from images and processing of the .json files outputted

JavaScript - Main operating language in React and backend Node.js which communicates with Microsoft Azure

HTML - Embedded XML within React

CSS - Styling within React

Challenges we ran into

The OCR algorithms supported less text than expected. Removing data from the database when deleting a text box was difficult to implement without encountering timing issues. Combining mobile and web development is new to all of us.

Accomplishments that we're proud of

Producing a finished and genuinely useful project in just 36 hours is an achievement in itself. Two of our team had never partaken in a hackathon and working remotely proved to be a challenge when helping each other.

What we learned

How to use APIs, set up a RESTful API, use databases, and implement a full-stack solution.

What's next for PaperSnap

Improving image processing algorithms to increase confidence intervals in our OCR algorithm, as well as the support of vectorization of hand-drawn diagrams to render on canvas. Increasing the number of characters that are detectable would also prove useful to some people, and a feature to solve basic maths equations autonomously would be interesting to implement. Improving workflow in BackEnd to allow seamless data flow across all platforms. Moving towards training our own algorithm.

Share this project:

Updates