Luma AI is a dementia care companion platform built for caregivers and patients. It combines AI agents, serverless workflows, and real-time monitoring to support memory assistance, medication reminders, wandering alerts, and MRI-based cognitive insights. The system is designed to run on DigitalOcean (App Platform, Functions, Spaces, Managed Postgres, Managed Valkey/Redis, OpenSearch).

Disclaimer
This is a research prototype, not a diagnostic medical tool. Do not use for real medical decision-making. All interfaces must display: "This is a research prototype, not a diagnostic medical tool."

• Features
• Architecture Overview
• Project Structure
• Prerequisites
• Getting Started
• Configuration & Environment
• Components in Detail
• Deployment
• Healthcare Disclaimer

┌─────────────────────────────────────────────────────────────────────────────┐
│                           Caregiver / Patient                                │
│                         (Browser → Django App)                               │
└─────────────────────────────────────────────────────────────────────────────┘
                                        │
                                        ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│  luma_web (Django)                                                           │
│  • Auth (caretaker vs patient), sessions, patient CRUD                       │
│  • Proxies: /agents/mri, /agents/memory/store, /agents/memory/search,       │
│    /agents/chat → DO Agents (BASE_URL + ACCESS_TOKEN)                        │
│  • Uploads to DO Spaces (memories, MRI studies)                              │
│  • Dashboard data from Postgres (wander_alerts, memory_retrieval_logs,      │
│    medication_schedules, mri_studies, usual_spots)                           │
└─────────────────────────────────────────────────────────────────────────────┘
         │                    │                    │                    │
         ▼                    ▼                    ▼                    ▼
┌──────────────┐    ┌─────────────────┐   ┌──────────────┐   ┌─────────────────┐
│ DO Agents    │    │ DO Serverless   │   │ DO Spaces    │   │ Managed         │
│ (Caregiver,  │    │ Functions       │   │ (S3-compat)  │   │ Postgres        │
│ Memory, Chat)│    │ (see below)     │   │              │   │                 │
└──────────────┘    └─────────────────┘   └──────────────┘   └─────────────────┘
         │                    │                                      │
         │                    │  memory_store, memory_retrieve,       │
         │                    │  mri_analysis, medication_worker,     │
         │                    │  risk_analyser                        │
         │                    ▼                                      │
         │           ┌─────────────────┐   ┌─────────────────┐       │
         │           │ OpenSearch      │   │ MRI Inference    │       │
         │           │ (embeddings,    │   │ (FastAPI+ONNX   │       │
         │           │  patient-memories)│   │  on GPU VM)     │       │
         │           └─────────────────┘   └─────────────────┘       │
         │                                                             │
┌────────┴────────┐                                                    │
│ Wandering       │  Valkey (Redis) streams ← simulator / GPS          │
│ detector        │  → Twilio SMS + INSERT into wander_alerts ─────────┘
└─────────────────┘

• Django is the main web app: auth, dashboard, patient management, and proxy to AI agents.
• DigitalOcean Agents (Caregiver, Memory, Chat) are invoked by Django via DO_*_AGENT_BASE_URL and DO_*_AGENT_ACCESS_TOKEN. Agents can call serverless functions and tools.
• Serverless functions handle: memory store/retrieve (Gemini + OpenSearch + Spaces), MRI pipeline (Spaces → inference URL → Postgres), medication SMS (Postgres + Twilio), risk summary (Postgres aggregations).
• MRI inference is a separate FastAPI service (e.g. on a GPU droplet) exposing /predict; the mri_analysis function calls it via INFERENCE_URL.

LumaAI/
├── luma_web/                    # Django application (main web app)
│   ├── luma_web/                # Project settings, urls, wsgi
│   ├── luma_app/                # App: models, views, services, templates
│   ├── requirements.txt
│   ├── env-example              # Example environment variables
│   └── Dockerfile               # For App Platform / GHCR
│
├── functions/                   # DigitalOcean Serverless Functions
│   ├── memory_store/            # Store memory (text/image → Gemini → OpenSearch)
│   ├── memory_retrieve/         # Semantic + keyword search, usual spots, logging
│   ├── mri_analysis/            # Fetch image from Spaces → inference → Postgres
│   ├── medication_worker/       # Cron: due meds → Twilio SMS, update last_sent_at
│   └── risk_analyser/           # Aggregates logs for caregiver agent (wander, memory, MRI)
│
├── mri_detection/               # Standalone MRI inference API (FastAPI + ONNX)
│   ├── app.py                   # /predict endpoint (ViT model)
│   ├── alzheimer_onnx/          # ONNX model (or restore from alzheimer_onnx_backup.tar.gz)
│   └── commands.txt             # Setup and curl examples
│
├── wandering_notice/            # Wandering alert pipeline
│   ├── simulator.py             # Pushes GPS events to Valkey stream (patient:2:gps_stream)
│   └── detector.py             # Consumes stream, geodesic distance, Twilio SMS + Postgres
│
├── agents/                      # Agent prompts / instructions (for Gradient/DO Agents)
│   ├── Luma_AI_Caregiver_Agent.md
│   ├── Luma_AI_Patient_Intelligence_Agent.md
│   └── Luma_AI_Supervisor_Agent.md
│
├── frontend/                    # Static HTML/CSS reference (dashboard, chat, memories, MRI, etc.)
├── KnowledgeBases/              # PDFs for agent context (e.g. NIA caregiver guides)
└── .github/workflows/           # CI (e.g. build and push luma_web to GHCR)

• Python 3.10+ (3.11 for luma_web Dockerfile)
• PostgreSQL (e.g. DigitalOcean Managed Database)
• Redis/Valkey (e.g. DO Managed Valkey) for wandering streams
• OpenSearch (for memory embeddings index patient-memories)
• S3-compatible storage (e.g. DO Spaces) for images and MRI studies
• Twilio account (SMS for medication reminders and wandering alerts)
• Google Gemini API key (embeddings and vision in memory store/retrieve)
• DigitalOcean Agents (or compatible OpenAI-style API) for Caregiver, Memory, and Chat
• doctl and DigitalOcean Serverless plugin for deploying functions

git clone <repo-url>
cd LumaAI

cd luma_web
python3 -m venv .venv
source .venv/bin/activate   # Windows: .venv\Scripts\activate
pip install -r requirements.txt
cp env-example .env
# Edit .env with your DATABASE_*, SPACES_*, and DO_*_AGENT_* variables
python manage.py migrate
python manage.py runserver

• App: http://127.0.0.1:8000
• Login/signup, then dashboard, patients, memories, MRI upload, chat.

Used by the mri_analysis function. Run on a machine with GPU for best performance:

cd mri_detection
python3 -m venv alz_env && source alz_env/bin/activate
pip install "numpy==1.26.4" optimum[onnxruntime] transformers fastapi uvicorn python-multipart Pillow torch torchvision accelerate
# If you have alzheimer_onnx_backup.tar.gz: tar -xzf alzheimer_onnx_backup.tar.gz
python app.py

• API: http://0.0.0.0:8000 (health: /, predict: POST /predict with [email protected]).
• Set INFERENCE_URL in the mri_analysis function environment to this host.

• Simulator (sends fake GPS to Valkey):

  cd wandering_notice
  pip install redis python-dotenv
  # Set VALKEY_URL in .env
  python simulator.py

• Detector (consumes stream, sends SMS and writes to Postgres):

  pip install redis psycopg geopy python-dotenv twilio
  # Set VALKEY_URL, DATABASE_URL, TWILIO_*, CAREGIVER_PHONE in .env
  python detector.py

From the root of each function (e.g. functions/memory_retrieve):

doctl serverless connect
doctl serverless deploy . --env .env --remote-build

Deploy each of: memory_store, memory_retrieve, mri_analysis, medication_worker, risk_analyser. Ensure their .env (or DO namespace env) has the required keys (e.g. DATABASE_URL, OS_*, SPACES_*, GEMINI_API_KEY, TWILIO_*, INFERENCE_URL for mri_analysis).

Copy luma_web/env-example to luma_web/.env and set:

Each function’s environment typically needs:

• memory_store / memory_retrieve: OS_HOST, OS_USER, OS_PASS, GEMINI_API_KEY, SPACES_*, DATABASE_URL (retrieve also logs to memory_retrieval_logs and uses usual_spots).
• mri_analysis: SPACES_*, DATABASE_URL, INFERENCE_URL (e.g. http://<mri-detection-host>:8000/predict).
• medication_worker: DATABASE_URL, TWILIO_SID, TWILIO_AUTH_TOKEN (and Twilio “from” number in code if different).
• risk_analyser: DATABASE_URL.

• simulator: VALKEY_URL
• detector: VALKEY_URL, DATABASE_URL, TWILIO_SID, TWILIO_AUTH_TOKEN, CAREGIVER_PHONE

• Models: Patient, MedicationSchedules, MemoryRetrievalLogs, MriStudies, UsualSpots, WanderAlerts (most unmanaged, mapping to existing tables).
• Views: Login/signup, dashboard, patient CRUD, memories page, MRI upload, chat. Agent proxies build prompts (with patient_id, image URLs, etc.) and call invoke_agent(prefix, payload) from services.py.
• Agent integration: services.invoke_agent(prefix, payload) uses {prefix}_BASE_URL and {prefix}_ACCESS_TOKEN, POSTs to an OpenAI-compatible /api/v1/chat/completions endpoint, returns markdown and raw JSON.

• memory_store: Input: patient_id, text_note, image_url. Fetches image from Spaces if present, describes with Gemini vision, builds unified text, embeds with Gemini, indexes in OpenSearch patient-memories.
• memory_retrieve: Input: patient_id, query. Embeds query, runs OpenSearch k-NN + keyword, merges caretaker “usual spots” from Postgres, logs to memory_retrieval_logs, returns combined results with optional presigned image URLs.
• mri_analysis: Input: patient_id, image_url. Fetches image from Spaces, normalizes, calls INFERENCE_URL for prediction, writes row to mri_studies.
• medication_worker: Runs on schedule; selects rows from medication_schedules where reminder time (in patient timezone) is due and not yet sent today, sends Twilio SMS, updates last_sent_at.
• risk_analyser: Input: patient_id, days. Returns recent memory retrievals, wander alerts, and latest MRI for the caregiver agent.

• FastAPI app loading an ONNX Vision Transformer from ./alzheimer_onnx, exposes POST /predict (image file) and returns prediction, confidence, and disclaimer text.

• simulator: Appends GPS-like entries to Valkey stream patient:2:gps_stream.
• detector: Reads stream, computes geodesic distance from SAFE_ZONE_CENTER; if > SAFE_RADIUS_METERS, sends Twilio SMS and inserts into wander_alerts.

• Django: The repo includes a Dockerfile under luma_web/. GitHub Actions (e.g. .github/workflows/publish-ghcr.yml) can build and push the image to GHCR; point DigitalOcean App Platform at that image and set env vars.
• Functions: Deploy with doctl serverless deploy . --env .env --remote-build from each function directory; configure triggers (e.g. cron for medication_worker) and wire agent tools to the function endpoints.
• MRI inference: Deploy mri_detection on a VM or GPU instance; set INFERENCE_URL in the mri_analysis function.
• Wandering: Run detector.py as a long-lived process (e.g. systemd or container) with access to Valkey and Postgres.

All interfaces and outputs in this project must clearly state:

"This is a research prototype, not a diagnostic medical tool."

Do not use this system for real medical decision-making. Ensure compliance with local regulations and clinical validation before any clinical or diagnostic use.

See repository license file.