Your Personal AI Healthcare Companion. Built for nwHacks 2026
Baymini is an autonomous IoT healthcare agent designed to protect users from severe allergic reactions and medication conflicts. By integrating a seamless web dashboard with a headless Raspberry Pi unit, Baymini acts as a second pair of eyes—scanning products, analyzing ingredients using Gemini Vision AI, and speaking safety warnings in real-time.
- 👁️ AI Vision Analysis: Uses Gemini 2.5 Flash to visually analyze product packaging and ingredient labels in milliseconds.
- 🗣️ Voice Interface: Provides immediate, spoken feedback using Google Text-to-Speech (TTS), acting as a caring companion.
- ☁️ Instant Sync: A modern Flask dashboard allows users to update allergies/meds, which are "beamed" instantly to the headless Pi unit via SCP.
- 🛡️ Personalized Care: Logic is dynamically updated based on the specific user profile (e.g., specific allergies to Peanuts or conflicts with Insulin).
- Raspberry Pi 4 (Headless Mode)
- USB Webcam
- Speaker / Audio Output
- AI Model: Google Gemini 2.5 Flash (via Google Generative AI SDK)
- Computer Vision: OpenCV (
cv2) - Backend: Python Flask
- Voice: Google Text-to-Speech (
gTTS) - Networking: SSH & SCP for local data synchronization
This project consists of two parts: the Web Dashboard (run on your laptop) and the Robot Client (run on the Raspberry Pi).
- Python 3.11
- A Raspberry Pi with SSH enabled.
- A Google Gemini API Key.
- User Input: The user logs into the Web Dashboard and updates their profile (e.g., "Allergy: Peanuts").
- Sync: The Dashboard generates a JSON profile and uses
scpto securely transfer it to the Raspberry Pi over the local network. - Scan: The user holds a product in front of the Pi and triggers a scan.
- Analyze: The Pi captures an image and sends it + the JSON profile to Gemini 2.0 Flash.
- Reasoning: The AI reads the ingredients list, checks against the specific user constraints, and generates a response.
- Response: The Pi speaks the result aloud (e.g., "Warning: This product contains peanuts.").