GuideMeSG

GuideMeSG empowers independent travel for people with intellectual disabilities using AR navigation and real-time caregiver monitoring.

Comment 5

GuideMeSG – Project Story

Inspiration

Public transit apps in Singapore are not designed for individuals with cognitive disabilities such as persons with intellectual disabilities (PWIDs) or early-onset dementia. Mainstream navigation tools are often too complex, stressful, and potentially dangerous for these users, who require simpler interfaces, real-time oversight, and immediate help options to travel safely.

We set out to build a system that restores independence while still giving caregivers peace of mind through real-time visibility and safety mechanisms.

What it does

GuideMeSG is an accessibility-first navigation web app built for cognitively vulnerable travelers and their caregivers.

For Travelers (Patients)

  • Stabilized AR Navigation
    A large, high-contrast green arrow overlays the live camera feed and points toward the next waypoint, reducing the need to interpret a 2D map.

  • Bus Crowd Safety Validator
    Before boarding, the system predicts crowd density (e.g., Low / Medium / High) using public transport data, helping avoid overwhelming situations.

  • Voice Navigation
    Step-by-step instructions are read aloud using text-to-speech to reduce cognitive load and reliance on reading.

  • Emergency “I’m Lost” Button
    One tap reroutes the user to the nearest safe “anchor” location (e.g., Dementia Go-To Point, hospital, or police post).

  • Call Guardian Button
    A dedicated button to immediately call their assigned caregiver.

  • Patient Profile Page with QR Code
    Displays essential information (name, caregiver contact, basic medical information). A QR code lets passers-by scan and see how to help and whom to call.

For Caregivers

  • Patient–Caregiver Linking System
    Caregivers and patients are paired via a simple code-based flow. Once linked, the caregiver can see key details and routes.

  • Deviation Alerts
    If the patient strays significantly off the intended route, the caregiver receives a live deviation alert and can see the patient’s last known path.

  • Live GPS Location and Map View
    Caregivers can monitor a live map with the patient’s location and intended route.

  • Live Camera Snapshot (Prototype)
    When AR navigation is active, the system can surface a snapshot of the patient’s front camera to give context (e.g., whether they look distressed or confused).

  • Patient Status Dashboard
    A dashboard summarises each patient’s current status (en route, arrived, deviated, idle) and last-known location.

  • Remote Destination Manager
    Caregivers can define and manage “safe” destinations (home, day centre, clinic). Patients can then simply choose from these instead of typing addresses.

  • Resizable Layout for Monitoring
    The caregiver interface uses a flexible layout (panels, split-view) to allow comfortable monitoring of multiple patients.

All of this forms an active safety loop: travelers navigate with a simple AR arrow, while caregivers retain high-level control and can intervene quickly if something goes wrong.

How we built it

GuideMeSG is implemented as a full-stack web application using modern JavaScript tooling and real-time infrastructure.

Stack and Architecture

  • Frontend

    • Next.js 14 (App Router, TypeScript)
    • React with Context for auth and shared state
    • Tailwind CSS and shadcn/ui for accessible components
    • React Leaflet for interactive maps (OpenStreetMap tiles)

  • Backend

    • Next.js serverless API routes deployed on Vercel
    • Route planning and deviation detection implemented as API endpoints
    • Middleware for authentication, authorization, and data validation

  • Database and Realtime

    • Supabase (PostgreSQL) to store:
    • Users (patients, caregivers)
    • Linked relationships
    • Saved destinations
    • Deviation events and audit logs
    • Supabase Realtime for pushing deviation alerts and updates to caregivers instantly

  • Device & AR Capabilities

    • MediaDevices.getUserMedia for camera access
    • DeviceOrientationEvent (gyroscope/compass) to orient the AR arrow
    • A custom stabilisation layer that smooths noisy orientation data
    • Speech Synthesis API for voice navigation

  • External Data and Integrations

    • Public transport API (e.g., bus arrival and crowd/load information)
    • Geocoding and routing (e.g., OneMap / mapping provider)
    • Public datasets of dementia-friendly “Go-To” points and related facilities

The system is structured as a set of clearly defined modules:

  • Navigation Engine (waypoint generation, AR bearing computation)
  • Deviation Engine (geofencing, off-route detection)
  • Realtime Alert System (Supabase listeners and UI notifications)
  • Profile and Pairing Module (authentication, linking caregivers to patients)

Challenges we ran into

  1. Compass and AR Stability
    Mobile browsers expose noisy orientation data. We had to implement smoothing and threshold logic so that the AR arrow feels stable instead of jittery.

  2. Realtime Coordination Between Two Roles
    Getting patient and caregiver data to stay in sync—especially for live location and deviations—required careful schema design and subscription management.

  3. Designing for Cognitive Accessibility
    Reducing cognitive load meant stripping away common UI elements (text-heavy maps, dense menus) and iterating toward:

    • Single primary actions on each screen
    • High-contrast, large UI elements
    • Clear visual hierarchy

  4. API Reliability and Latency
    Transport and map APIs can be slow or rate-limited. We added basic caching strategies and fallback messaging to keep the experience responsive.

  5. Balancing Detail vs. Overload for Caregivers
    We needed to show enough data (route, status, alerts) without overwhelming caregivers with noisy notifications or cluttered dashboards.

Accomplishments that we are proud of

  • Built a working stabilized AR guidance system in the browser, without requiring a native app.
  • Implemented real-time deviation alerts using Supabase Realtime and structured geofencing logic.
  • Designed and implemented a full patient–caregiver ecosystem:
    • Pairing, profiles, and linking
    • Remote destination configuration
    • Live monitoring and safety interventions
  • Created an interface that follows accessibility-first principles, with simple flows and clearly highlighted primary actions.
  • Integrated real transport and mapping data to support:
    • Crowd-aware bus choice
    • Safe rerouting to known anchor points
  • Delivered a solution that can realistically plug into existing caregiving workflows and dementia/day-care centres.

What we learned

  • Cognitive accessibility is different from general accessibility.
    It is not just about font sizes or contrast, but about minimising choices per screen and avoiding ambiguous states.

  • Realtime systems are as much about UX as they are about websockets.
    Even with low-latency updates, if alerts or state changes aren’t prioritised and summarised clearly, caregivers feel overwhelmed.

  • The web platform is surprisingly capable for AR-like experiences.
    With careful handling of orientation, camera, and voice APIs, we can prototype AR guidance without an app store.

  • Working with public APIs requires defensive design.
    Any external API can fail, lag, or return edge cases; resilient fallbacks are essential for safety-critical workflows.

  • Collaboration across roles matters.
    Combining perspectives from UX, accessibility, and systems engineering was critical to avoid building a “cool” demo that is unusable in real caregiving contexts.

Impact

GuideMeSG empowers individuals with cognitive disabilities to navigate public transport more safely and independently through simplified AR guidance and real-time support. By reducing confusion, preventing wandering, and enabling immediate caregiver intervention, the system enhances user confidence while significantly lowering caregiver stress. This solution strengthens inclusive mobility in Singapore and provides a scalable safety net that supports families, care centres, and the wider community.

Built With

+ 23 more
Share this project:

Updates