Inspiration
We were inspired to provide a new approach to accessibility. While traditional automation incorporates verbal cues from the user, our idea builds off of the common issues for neurodiverse individuals - such as those with autism, ADHD, or speech impediments - by basing the response on user motions, which can create a stress-free environment for these people.
What it does
HelpingHome acts as an accessibility layer for any home, operating in three main areas: the bathroom, the kitchen, and the laundry room.
- The Kitchen: Guides users through recipes step-by-step and monitors for dangerous situations, such as a hot stove left unattended or loud noises that could trigger sensory overload, automatically dimming lights, and playing calm audio to de-escalate stress.
- The Laundry Room: Uses vibration analysis to track wash cycles and provides calm reminders to prevent wet clothes from being forgotten and spoiling.
- The Bathroom: Uses thermal sensors to prevent scalding by warning users if the water is too hot before they touch it. Step-by-step instructions are given for the hand washing protocol, and reminders for when the user is taking a break.
How we built it
We built a Python module-based system backed by the Indistinguishable From Magic hardware. We integrated thermal sensors for temperature tracking, proximity/distance sensors for presence detection, motion sensors for safety monitoring, a force level sensor for vibration checks, and LED lights with color based on the event. We coded different state machines for each room (kitchen.py, bathroom.py, and laundry.py) to handle the complex logic (like distinguishing between a person washing hands and a tap left running, or determining if a laundry cycle has started from force sensors). We integrated ElevenLabs to generate calm and natural voice cues that reduce stress and anxiety. We also built a logging system using OpenNote, which sends raw sensor data through OpenNote’s API, where it then processes and summarizes activities, allowing caregivers to monitor routines easily and efficiently.
Challenges we ran into
Our biggest challenge was Hardware-API Integration. Getting the Indistinguishable From Magic sensors to reliably communicate with our Python backend required reverse-engineering the flow of data through our system. We also struggled to keep logic consistent. Ensuring that the “safety logic” felt consistent across three different rooms with different sensor data was difficult, as we knew that each room would consist of different logic, tailored to its specific issues.
Accomplishments that we're proud of
We are incredibly proud of our “invisible interface”. We managed to create a system that helps users without requiring any touch screen or voice commands. We successfully repurposed simple hardware for complex solutions, such as using a vibration sensor to detect the subtle shaking of a washing machine cycle. Getting these sensors to work in harmony to solve real problems humans face is our greatest accomplishment.
What we learned
We mastered Python API integration (ElevenLabs, OpenNote, IFMagic), debugging of hardware issues, and processing signals. We utilized Google AI Studio and Premiere Pro for documentation and presentation as well. We have applied our understanding of logic and honed our problem-solving skills by reverse engineering the sensor signals to behave properly.
What's next for HelpingHome
We plan to expand our reach beyond just these three rooms. Our next target is the Bedroom, in which we will implement sleep hygiene instruction and morning routine sequences.
Log in or sign up for Devpost to join the conversation.