Luma Ai

Empowering Families with AI-Driven Dementia Care

Comment

Inspiration

Dementia affects more than memory — it reshapes daily life for both patients and caregivers. Caregivers often become what researchers call “shadow patients”, carrying the emotional and cognitive burden of constant vigilance: remembering medications, ensuring safety, and interpreting medical signals.

Today’s tools are fragmented. GPS trackers operate independently from medical insights. Memory notes exist separately from care plans. Clinical imaging rarely translates into daily actionable guidance.

We built Luma AI to bridge this gap.

Luma AI is designed as a holistic AI companion for dementia care, combining real-time monitoring, AI memory assistance, clinical signal analysis, and caregiver intelligence into a single platform.

Our goal is simple: make high-quality dementia support accessible, proactive, and intelligent — for every caregiver and patient.

What it does

Luma AI is an AI-powered dementia care platform that supports patients and caregivers through five intelligent capabilities.

1. Memory Vault – A Semantic “Second Brain” 🧠

Patients can store memories as text notes or photos (e.g., “Left my glasses on the kitchen counter”). These memories are converted into embeddings and stored in a vector index, enabling natural language recall.

Instead of searching manually, patients can ask questions like:

“Where did I keep my house keys?”

The system performs semantic retrieval and merges results with caregiver-defined “usual spots”, helping patients rediscover objects and reduce anxiety.

Key capabilities:

  • Semantic memory storage
  • Natural language retrieval
  • Image + text memory support
  • Caregiver-defined memory anchors

Memory Addition Flow

Memory Addition Flow

Memory Retrieval Flow

Memory Retrieval Flow

2. Wandering & Safety Monitoring 📍

Wandering is one of the most dangerous dementia behaviors. Luma AI provides real-time safety monitoring using location telemetry.

GPS events are streamed through a real-time event pipeline powered by Valkey streams. A dedicated detector continuously calculates the patient’s distance from a predefined safe zone.

If the patient moves outside the defined radius:

  • The system triggers instant alerts
  • Caregivers receive Twilio SMS notifications
  • Events are logged for risk analysis

This pipeline processes events continuously and can react within seconds.

Wandering Detection Flow

3. Clinical Intelligence – MRI Analysis 🧬

Caregivers can upload MRI scans directly to the platform.

These scans are stored in DigitalOcean Spaces and processed by a Image Classification Model inference service.

The model analyzes the MRI for potential cognitive markers associated with neurodegenerative diseases.

Instead of returning raw model predictions, our AI caregiver agent interprets the results and provides clear summaries such as:

  • Observed cognitive risk signals
  • Confidence scores
  • Suggested next steps for medical consultation

This bridges the gap between complex clinical imaging and understandable caregiver insights.

MRI Analysis Flow

4. Medication Reminder System 💊

Medication adherence is critical in dementia care, yet it is often difficult to maintain consistently.

Luma AI includes a fully automated medication reminder system powered by DigitalOcean serverless triggers.

The Medication Worker is a scheduled DigitalOcean Function that runs in the background and performs the following workflow:

  1. It reads all active medication schedules from PostgreSQL.
  2. For each row in medication_schedules, it checks the stored patient timezone.
  3. It determines whether the reminder_time has passed for the current local day.
  4. It verifies the last_sent_at field to ensure reminders are not sent twice.

If a medication dose is due:

  • The system generates a caregiver-friendly SMS reminder
  • The message includes medication name and dosage
  • The SMS is sent through Twilio
  • last_sent_at is updated to prevent duplicate reminders

This creates a lightweight, timezone-aware reminder system that runs entirely in the background without requiring the web application to remain online.

5. Caregiver Intelligence Dashboard 📊

Caregivers receive a unified dashboard that aggregates patient health signals.

The system analyzes multiple behavioral indicators, including:

  • Wandering events
  • Memory retrieval success/failure
  • Medication adherence
  • MRI analysis results

These signals feed into a risk analysis engine that highlights patterns such as:

  • Increasing wandering frequency
  • Declining memory recall performance
  • Potential cognitive risk escalation

This enables proactive care rather than reactive responses.

Risk Analysis Flow

How we built it

Luma AI is built as a cloud-native multi-agent architecture running entirely on the DigitalOcean ecosystem.

Core Web Platform

The main application is built using Django and deployed on DigitalOcean App Platform, which allowed us to quickly deploy a production-ready backend without managing infrastructure.

Our deployment workflow works as follows:

  1. A GitHub Actions workflow builds a Docker image for the Django app.
  2. The image is pushed to a container registry.
  3. DigitalOcean App Platform automatically deploys the image, handling scaling, networking, and environment configuration.

This allowed us to focus entirely on building product features rather than managing servers.

Multi-Agent AI Architecture 🤖

Luma AI uses a Supervisor Agent architecture.

A central Supervisor Agent routes user requests to specialized agents:

  • Patient Intelligence Agent → handles semantic memory storage and retrieval
  • Caregiver Agent → analyzes MRI scans and patient signals and also generates insights and summaries for caregivers

This modular approach makes the system scalable and easier to extend with additional intelligence modules.

Multi Agent Architecture

Serverless AI Workflows

We used DigitalOcean Functions extensively to power event-driven AI workflows.

Serverless functions handle:

  • Memory embedding generation
  • Semantic memory retrieval
  • MRI analysis pipelines
  • Medication reminder scheduling
  • Risk signal aggregation

Because these workflows are serverless, they scale automatically and run only when needed, reducing operational overhead.

Data Infrastructure

Luma AI integrates several specialized data services:

Component Purpose
Managed PostgreSQL structured patient data and analytics
OpenSearch vector search for memory embeddings
Managed Valkey (Redis) real-time GPS telemetry streams
DigitalOcean Spaces storage for memory images and MRI scans

AI & Inference Layer

Our AI stack combines multiple models and services:

  • Gemini embeddings for semantic memory understanding
  • Image Classification (ONNX) for MRI image analysis
  • Open AI for LLM summarisations

This allows Luma AI to combine LLM reasoning, computer vision, and real-time analytics in one platform.

Challenges we ran into

Semantic Memory Ambiguity

Vector search sometimes returned semantically related but incorrect results (e.g., retrieving “glasses” when searching for “keys”).

We solved this by implementing Hybrid Search, combining vector similarity with BM25 keyword ranking, improving accuracy significantly.

Timezone-Aware Medication Scheduling

Medication reminders required correct scheduling across global timezones.

We solved this by moving timezone logic into PostgreSQL queries using AT TIME ZONE, ensuring reminders trigger correctly regardless of location.

Network Restrictions for Notifications

SMTP email alerts initially failed due to blocked ports on our cloud instance.

We pivoted to Twilio’s HTTPS API, ensuring reliable notifications over port 443.

Containerizing Native Dependencies

Building Docker images for PDF generation required native libraries like pycairo.

We resolved this by adding required Linux headers and system packages in the Dockerfile to ensure stable builds.

Accomplishments that we're proud of 🎉

Real-Time Safety Monitoring

We built a live telemetry pipeline that streams GPS events into Valkey and triggers caregiver alerts within seconds.

End-to-End AI Medical Pipeline

MRI scans can be:

  1. Uploaded by caregivers
  2. Processed by model inference
  3. Stored in the database
  4. Interpreted by AI agents

All within a fully automated workflow.

Context-Aware AI Agents

The chat interface intelligently adapts responses depending on whether the user is a patient or caregiver, improving usability and empathy.

Cloud-Native Architecture

By combining App Platform, Functions, Managed Databases, Spaces, and OpenSearch, we built a scalable AI platform entirely within the DigitalOcean ecosystem.

What we learned

Building Luma AI taught us several key lessons.

First, AI systems in healthcare must prioritize reliability over novelty. Semantic search alone is not enough — combining vector and keyword search dramatically improves trust.

Second, event-driven architectures are extremely powerful. Serverless functions allowed us to build complex AI workflows without managing infrastructure scaling.

Finally, multi-agent architectures simplify AI system design, enabling each agent to focus on a specific responsibility while the supervisor coordinates the overall system.

What's next for Luma AI

We see Luma AI evolving into a full AI-powered dementia care ecosystem.

Wearable Integration

Integration with devices such as Apple Watch and Fitbit to monitor vitals, activity levels, and sleep patterns.

Computer Vision Memory Assistance

Patients could ask:

“What is this?”

Luma AI would analyze live camera feeds to recognize objects or faces stored in the Memory Vault.

Predictive Risk Intelligence

By analyzing historical wandering patterns, medication adherence, and memory retrieval data, Luma AI could predict high-risk days and notify caregivers in advance.

Clinical Validation

Future work includes collaboration with medical researchers to validate the MRI model and behavioral analytics.

Disclaimer

Luma AI is a research prototype and not a diagnostic medical tool. It is designed to assist caregivers and researchers and should not replace professional medical advice.

Built With

Share this project:

Updates