Oculo

Our logo

Inspiration

While talking with our friend from the US about differences between our country, Canada, and theirs, we found out US bills don't have braille like Canadian bills. It was so surprising to us and we wondered how in the world blind people in the states manage their money. So we did some research and found out Orbis reports 43 million people globally live with blindness. For these people, out of Ranker's 14 most common problems they face when being blind, two of them involve money. Thus, we created an application to allow the visually impaired to identify money amounts. Users will use a phone app that classifies money bills and reads out money bill values.

What it does

Our application uses deep learning and computer vision to process dollar bill images and classify them. We then use text-to-speech to read out the classification and dollar bill amount out loud to users. Our interface is extremely simple and tethered towards blind people. The app is also available on multiple devices like computers and mobile phones.

How we built it

We built our model by querying thousands of images using Microsoft Azure's Bing Search API. We then augmented the data and trained it using fast.ai. The model was then deployed onto our flask API to allow multiple applications and devices to use our model. The frontend was built with Vue.js and Python to support multiple different platforms.

Challenges we ran into

One of our biggest challenges was dealing with the different file formats and sending/receiving them from our servers and clients. Our app dealt with both audio and image files and it was challenging to get this part working.

Accomplishments that we're proud of

Our team consists of pretty inexperienced programmers. For many of us, it was our first time experimenting with machine learning or building APIs. However, we worked very hard and learned so much from building this project. We are extremely proud that we were able to accomplish the project by the Hackathon deadline given our low amount of experience.

What we learned

We learned that no challenge is too great as long as we're determined to learn and grow. Through this project, some of us who had very little experience in programming learned a great deal about frontend, backend, machine learning, and working with third-party APIs.

What's next for Oculo

Our application was built to be very easily scalable. We believe the possibilities for Oculo are endless. Our app is able to identify and classify anything from food to traffic signals for blind people. To add more classification it's as simple as feeding more data into our model. Almost no code needs to be changed when adding new classifications.