CapVision: Empowering Navigation for the Visually Impaired

Every year at HackNotts, my goal is to create something that helps individuals with impairments better understand the world around them. This year, I focused on developing a solution for people with visual impairments, leading to the creation of CapVision.

What It Does

CapVision is a mobile device that leverages real-time object detection to provide crucial audio feedback, alerting users to nearby obstacles such as people, chairs, and tables. Imagine wearing a baseball cap that acts as an extra set of eyes, verbally guiding you through your environment. This system aims to enhance independent navigation and increase safety for visually impaired individuals in various settings.

How We Built It

The prototype for CapVision was constructed using readily available and portable components:

• Raspberry Pi Model 3B: This served as the core processing unit, handling the object detection algorithms.
• A Power Bank: Ensuring the device's portability and continuous operation without being tethered to a power outlet.
• A Webcam: Mounted discreetly on the cap, the webcam captures the visual input for real-time analysis.
• A Baseball Cap: The chosen form factor for its wearability and ability to position the camera effectively for a user's perspective.

Challenges We Ran Into

Developing CapVision presented several significant hurdles, primarily concerning the model's accuracy and processing speed. The Raspberry Pi 3B, while versatile, is a relatively low-power device, which limited the complexity and efficiency of the computer vision model we could run in real-time. Furthermore, relying on a pre-defined, unfine-tunable model meant we couldn't optimize it for our specific use case or retrain it with more relevant data, impacting its precision and ability to distinguish between various obstacles effectively.

Accomplishments That We're Proud Of

Despite the challenges, we are immensely proud of creating a viable prototype. Seeing CapVision successfully detect objects and provide audio alerts, even in its early stage, was a powerful validation of our concept and the hard work invested. It demonstrated that a low-cost, portable solution for real-time obstacle detection is achievable.

What We Learned

This project was a deep dive into the practical application of Computer Vision Models and Computer Vision overall. We gained valuable hands-on experience in implementing object detection on embedded systems. A particularly insightful area of learning was Camera Depth Calculation, understanding how to infer the distance of detected objects from the camera, which is crucial for providing timely and accurate warnings to the user.

What's Next for CapVision

The future of CapVision is bright and holds immense potential for further development and refinement. Given more time and resources, our next steps would focus on significantly enhancing the system's capabilities:

• Custom Dataset Creation and Model Fine-Tuning: This is the most critical next step. Creating our own dataset with diverse real-world scenarios, specifically focusing on obstacles relevant to visually impaired individuals (e.g., street furniture, varying terrains, different lighting conditions), would allow us to train and fine-tune a custom computer vision model. This would dramatically improve the model's accuracy, reduce false positives, and increase its ability to recognize a wider array of objects.
• Expanding Object Recognition: Beyond people, chairs, and tables, we would prioritize adding more critical obstacles such as walls, curbs, stairs, uneven surfaces, and even hanging objects (like low branches or signs) that pose a significant hazard.
• Enhanced Depth Perception and Spatial Audio: Integrating more sophisticated depth sensors (e.g., a small LiDAR module or stereoscopic cameras) would provide more precise distance measurements. This data could then be used to implement spatial audio cues, where the sound of an approaching obstacle comes from the direction it's located, providing a more intuitive and immersive warning.
• Vibrational Feedback: In addition to audio, incorporating haptic feedback (vibrations) into the cap or a wearable wristband could offer an alternative or supplementary alert mechanism, especially in noisy environments.
• Optimizing for Low-Power Devices: Exploring more efficient model architectures or model quantization techniques would allow for better performance on resource-constrained devices like the Raspberry Pi, improving both speed and battery life.
• User Interface and Customization: Developing a simple companion mobile app could allow users to adjust settings, select preferred alert sounds, and potentially even receive a visual representation of detected objects for sighted companions.
• Durability and Weatherproofing: Designing a more robust and weather-resistant enclosure for the electronics would make CapVision suitable for outdoor use in various conditions.

CapVision has demonstrated the feasibility of using readily available technology to create a valuable assistive device. With continued development, it could evolve into an indispensable tool, significantly enhancing the independence and safety of visually impaired individuals.

Built With

• python