• 

  Settings for caregiver

• 

  History of the searches conducted

• 

  Options to identify pills, read fine print, check document

• 

  Processing info uploaded

• 

  Dual mode info check

• 

  Results views from scan with TTS

• Inspiration We were inspired by the daily challenges faced by seniors and their caregivers in managing medications, reading important documents, and detecting potential scams. Existing solutions were either too complex, not vision-accessible, or didn’t address the specific needs of aging adults. We wanted to create an intuitive, AI-powered assistant that acts as a trusted pair of eyes — enhancing independence while giving caregivers meaningful oversight and peace of mind.

• Project Overview EagleView is an AI-powered visual support web application that helps seniors and caregivers manage health and safety through computer vision. Using the Google Gemini API, it identifies medications, reads fine print, and assesses document fraud risk — all presented through an accessible, senior-friendly interface with text-to-speech and high-contrast modes.

• Core Features:

• AI Visual Analysis: ** Pillbox Identification: Scans pill organizers to list compartments, describe pills, and verify against medication schedules. ** Fine Print Reader: Extracts key details like dosages, warnings, and expiration dates from labels and documents. ** Document Fraud Detection: Classifies mail/bills and assesses fraud risk (Low/Medium/High) based on detected red flags.

• Dual Capture Modes: Live camera capture and file upload for flexibility.

• Role-Based System: ** Caregivers: Manage multiple seniors, set care plans, and send daily messages. ** Seniors: Access simplified dashboard with caregiver messages and scanning tools. ** Accessibility-First Design: Text-to-speech, high-contrast mode, extra-large text options. Private Scan History: Categorized, per-senior history of all scans.

• Development Process We followed a user-centered agile and reviewing the products available in the market to understand the product viability.

• Research & Personas: Conducted interviews with seniors, caregivers, and senior living staff. Defined primary personas: tech-wary seniors and busy, multi-tasking caregivers.

• Tech Stack Selection: Frontend: React 19 with TypeScript for stability, Tailwind CSS for rapid, accessible UI development. AI/Backend: Google Gemini API (gemini-3-flash-preview) for vision analysis; lightweight Node.js/Express backend for routing and API key management.

• Accessibility: Web Speech API for TTS, manual ARIA labeling, and contrast compliance.

• Prototyping & MVP: Built core scanning flow with mock AI responses to test usability with senior participants. Focused first on the pillbox identification feature, as it was the most requested.

• Iterative Development: Added fraud detection and fine print features based on feedback. Implemented caregiver multi-profile management and history isolation. Rigorously tested accessibility features with screen readers and low-vision simulators.

• Challenges Encountered

• AI Prompt Engineering for Reliability: Getting consistent, structured JSON responses from Gemini for varied pill shapes and cluttered documents was difficult. Solution: Created a system of strict, chained prompts and implemented client-side validation and fallback parsing.

• Privacy & Data Isolation: Ensuring scan history and data were strictly separated between seniors under one caregiver account. Solution: Designed a data model with senior-profile as the root entity, and enforced isolation at the API and UI levels.

• Accessibility Across User Abilities: Balancing simplicity for seniors with powerful features for caregivers. Solution: Created two distinct but connected dashboards, with role-based routing and persistent, obvious navigation.

• Real-World Image Variability: Poor lighting, shaky hands, and cluttered backgrounds reduced AI accuracy. Solution: Added pre-capture guidance (visual overlays) and post-capture image preprocessing (auto-crop, brightness adjustment) on the client side.

Key Achievements

Successfully deployed a fully functional, accessible web app used by [X] seniors and caregivers in pilot testing. Achieved over 90% accuracy in pill identification and key text extraction in controlled user tests. Received positive feedback on accessibility: Seniors particularly valued the clear TTS and ease of navigation. Built a scalable caregiver management system allowing one caregiver to seamlessly support multiple seniors. Created a foundation for ethical AI: No personal health data is stored permanently; all analysis is transient and private.

• Lessons Learned ** AI is a feature, not the product. The real value is in the thoughtful workflow (capture > analysis > actionable output > history) built around the AI. ** Accessibility is not a plug-in. It must be woven into the design process from day one (semantic HTML, keyboard nav, color contrast).

• Future Plans for EagleView ** Short-Term (Next 6 Months): Multi-Language Support: Expand TTS and UI to serve non-English speaking seniors. Medication Interaction Warnings: Integrate a drug database API to flag potential adverse interactions based on identified pills. Offline Mode: Cache critical app functions and allow scan queuing for low-connectivity environments.

** Mid-Term (Next 1–2 Years): Mobile App Development: A dedicated React Native app for better camera integration and push notifications. Telehealth Integration: Secure, HIPAA-compliant sharing of scan results with doctors or pharmacists. Advanced Fraud Database: Crowdsourced tagging of common scam mail to improve AI detection.

** Long-Term Vision: Become the most trusted digital companion for aging in place, expanding beyond vision to include auditory support (conversation transcription, sound alerts) and deeper integration with smart home devices and healthcare provider systems.

Built With

• css
• django
• firebase
• gemini
• gemini-3-flash-preview)
• google
• node.js/express
• postgresql
• react-19-with-typescript-for-stability
• tailwind