Inspiration

Over 1.5 billion seniors worldwide live alone or far from their families. When a student leaves for university in Toronto while their parents stay in India, a simple fall or a missed medication can turn into a tragedy — not because no one cared, but because no one knew.

We built ElderHarmony (SeniorCare AI) to be the always-on guardian between those two worlds. Not an alert system that spams families. Not a tracker that invades privacy. A quiet, intelligent layer that only speaks up when it matters — and acts when it must.

What It Does

SeniorCare AI monitors a senior citizen 24/7 through their Samsung Galaxy Watch or Whoop wearable. Every health data submission triggers a multi-agent AI pipeline that:

Watches vitals — heart rate, SpO2, HRV over 24 hours Detects falls — and places an emergency Twilio voice call + SMS to family within seconds, no human required Tracks medications — adherence windows (8AM / 2PM / 8PM), auto-refill when pills run low or 3-day miss detected Reads mood — EmoCare agent scores 1–5; low mood triggers a family wellness nudge Monitors activity — flags prolonged inactivity (>4h) as a welfare concern Builds family dashboards — remote family sees vitals, alerts, call history, and weekly AI summaries The family app and senior app are designed for their respective users: large text and PIN-only login for seniors; full dashboard, dosage log, and call history for family.

How We Built It

Backend — Python on AWS

Two backend implementations run side-by-side:

Lambda-based (original): Orchestrator uses HealthAnalyzer (deterministic clinical rules) + OpenRouter AI → event_router → async Lambda invocations per agent Railtracks agentic (primary): 9 specialized agents as tools — VitalSync, Calling, Medicine, EmoCare, Medication, Refill, Activity, Sleep, HealthRecords — all coordinated by an Orchestrator using the agents-as-tools pattern The data contract is a HealthPayload dataclass with clinical thresholds baked in (HR 50–120 bpm, SpO2 <92% critical, HRV <40% = possible infection, sleep <6h deficit).

We also built a Flask HTTP server (server.py) that runs the full deterministic pipeline locally — real Twilio calls fire on every critical event during the demo.

Twilio Integration

Fall detected → emergency_escalation() → simultaneous voice call + SMS to family, no delay Low mood / low HRV / vital anomaly → call_senior() + alert_family_sms() with smart family rotation (least-contacted first) All calls gracefully simulate when credentials are missing Frontend — Flutter

13 screens targeting Android + Web. Built with Flutter + Riverpod + go_router:

Senior app: PIN keypad login, big-text vitals, medication tracker with pill bars, AI incoming call screen with pulsing animation, dose history calendar, emergency screen Family app: health dashboard, 4-type alert feed (critical / warning / AI action / info), weekly report Web dashboard: 5 metric cards, dosage log, call history, agent results debug view The UI is scenario-driven — 9 synthetic health scenarios (normal day → critical alert → fall → declining week) switchable live for demo purposes.

Challenges We Faced

LLM tool-calling failure: Our primary LLM (GPT-OSS 120B via HuggingFace) received the full orchestrator system prompt but made zero tool calls — just echoed the health JSON back. Diagnosed via Railtracks session logs. Fixed by building a deterministic Flask server that bypasses the LLM entirely for routing. Twilio trial restrictions: Trial accounts can only call verified numbers. Family contact phone numbers in scenario data were placeholder values — calls silently failed. Fixed by routing all Twilio calls through the verified env number while preserving contact names for rotation display. Dual backend architecture: Keeping Lambda-based and Railtracks implementations in sync without breaking shared services (TwilioService, HealthAnalyzer, PharmacyAPI). Flutter ↔ Python bridge: AgentSimulator.dart mirrors all backend thresholds in pure Dart for offline/fallback use; the real server provides live data with automatic fallback.