Baymini

Picture of Baymini
Picture of Baymini

Inspiration

Inspired by the admired companion from Big Hero 6, we decided to try and build our own mini version of Baymax, Baymini. In the real world, managing health conditions is anxiety-inducing. For people with severe allergies or complex medication schedules, a simple trip to the grocery store can be dangerous. We built Baymini to be that second pair of eyes as an autonomous, vision-powered agent that knows exactly who you are and keeps you safe.

What it does

Baymini is comprised of 2 part ecosystem:

The Patient Website: A modern web dashboard where the patient can input their medical profile (allergies, chronic conditions, and current medications).
Baymini Unit: A Baymax plush connected to a Raspberry Pi 4 equiped with a camera and speaker and microphone. Once the user creates a profile, the data is put securely to the Baymini unit. The user simply holds a product (food, medicine, etc.) in front of Baymini. The robot uses computer vision to read the label, analyzes the ingredients against the user’s specific health constraints, and verbally warns them of dangers or confirms safety, (unfortunately in Alexa's voice)

How we built it

The Brain (AI): We usedGoogle Gemini 2.0 Flash (via OpenRouter). This model was perfect for its speed and multimodal capabilities. The Body (Hardware): A Raspberry Pi 4 running in headless mode acts as the physical agent. It uses OpenCV for image capture and Alexa for voice return. The Portal (Software): The dashboard is built with Python Flask. We designed a custom UI using CSS variables and animations to give it a polished feel. The Link: The link was built through Gemini-API prompt to look through health profiles of the currently logged in user.

Challenges we ran into

Figuring out the hardware implementation with the Raspberry Pi 4 was difficult. We struggled for a long time figuring out how to connect the API with the RP4, as well as giving it the ability to speak and listen to the user as well. We also struggled with figuring out how to have the RP4 hold the program without being pluggied into the laptop to give Baymini a more portable feature.

Accomplishments that we're proud of

Syncing: When the user enter new information on the website, Baymini immediately knows about the new profile instantly. The UI: We stepped away from basic HTML and built a fully responsive, animated interface that looks like a shipping product. Real World Utility: Seeing Baymini correctly identify a hidden peanut allergen in a candy bar for the first time was a huge win.

What we learned

Hardware Integration: We learned how to bridge the gap between a high-level web app and low-level Linux hardware commands. Multimodal AI: We gained a deep appreciation for the power of Vision LLMs. We learned that providing context alongside the image yields significantly more accurate results than analyzing the image in isolation.

What's next for Baymini

3D model and accurate voicing: Giving Baymini a 3d model and accurate voice will give it a more lifelike feel, allowing users to truly feel like they are immersed in the experience. More Sensors: Adding temperature and heart rate sensors to allow Baymini to perform basic triage. Mobile App: Porting the web dashboard to a native React Native app for on-the-go profile management.