BlindSpot

Inspiration

According to the World Health Organization, approximately 90% of visually impaired individuals reside in low- to middle-income regions. That's an overwhelming majority who cannot afford treatments or solutions due to their financial situations.

What it does

Our project BlindSpot is a cost-effective and privacy-focused solution to visual impairments thanks to its low production cost and local hosting. The cost of materials we used totals out to be nearly 20 times cheaper than standard CV hardware like the Meta Ray-Bans, which are upwards of $300. This cost difference is invaluable to an overwhelming majority of individuals across the globe who need this technology.

How we built it

To get the camera and microphone running accordingly, we utilized technologies such as ESP32 and Arduino-oriented C. We exposed the ESP camera feed via an HTTP endpoint and queried it. To acquire speech, the ESP queried the user's phone.

For the demonstration app, we utilized Typescript within React Native.

Challenges we ran into

One of the most notable challenges we faced was the audio stream processing. When attempting to record the .wav files of microphone recordings, they initially were unusable static. We eventually fixed that issue by utilizing a dedicated RTOS instead of a superloop for handling the processing.

Accomplishments that we're proud of

We're proud of being able to reach the extent of our prototype with the resources we had.

What we learned

We learned that hardware-focused projects require more preparation than someone with a software-heavy background would typically expect. There's lots of parts to keep in mind from the start, and we could have gotten things done earlier had we made sure that all of our hardware was ready to go.

What's next for BlindSpot

• AMD-trained models to distinguish individuals approaching/distancing from wearer.
• CV model visualizations via WebSpatial and HMDs such as the Pico 4 and Apple Vision Pro.
• Mass-manufacturing procedures to reduce production costs even further

Sources: https://www.who.int/news-room/fact-sheets/detail/blindness-and-visual-impairment https://cdn.who.int/media/docs/default-source/universal-health-coverage/who-uhl-technical-brief-vision.pdf?sfvrsn=9fd2fa9c_3&download=true

Built With

• arduino
• esp32
• react-native