A distributed operating system for real-time interactions between wearable devices under an Internet of Bodies framework. Connects body-worn sensors, AR glasses, and AI into a unified signal bus where your body state drives shared perception.

Built by Delightful.vision. Currently integrated with Snap Spectacles and a custom piezo collar, with a platform-agnostic XR layer supporting Quest, Vision Pro, and WebXR.

DelightfulOS proposes that the most meaningful AI interfaces are not screens or speakers, but the body itself — sensed, augmented, and shared.

A piezo contact microphone on your throat detects speech intent ~200ms before you make a sound. AR overlays on Spectacles show others your body state in real time. Tapping someone's collar changes what you see over them. The collar is a physical interface that others use to shape their experience of you.

Core capabilities:

• Real-time body state estimation from piezo contact microphone signals
• Bidirectional audio AI via Gemini Live (listen, transcribe, respond, summarize)
• AR overlays and social cues pushed to Snap Spectacles via Supabase Realtime
• Rule-based and LLM-powered social mediation (turn-taking, stress, interruptions)
• Signal-reactive policies for low-latency physical interactions (collar tap -> AR)
• Modular Hardware Description Language for wearable co-design with AI
• Multi-user, multi-device — every user gets independent state tracking

See docs/VISION.md for the full conceptual and technical position.

Layered monorepo inspired by ROS (typed pub/sub topics) and MentraOS (manager-based composition):

delightfulos/              Installable Python package
  os/                      Core OS primitives (zero external deps)
    types.py                 Signal, Action, DeviceInfo, DeviceType, Capability
    registry.py              Device registry (what's connected)
    bus.py                   Signal bus (pub/sub, ROS-style topics)
    state.py                 State estimator (per-user body state)
  runtime/                 Orchestration layer
    managers.py              Runtime, DeviceManager, PolicyManager
    policy.py                Rule-based policy engine
    output.py                Output router (action -> device)
  ai/                      AI stack
    config.py                Settings (pydantic-settings, env vars)
    prime.py                 Prime Intellect inference client (OpenAI-compatible)
    mediator.py              AI social mediator (LLM-based policy)
    gemini_live.py           Gemini Live bidirectional audio sessions
    signal.py                Piezo signal processing (VAD, features)
    transcribe.py            Audio transcription via Gemini
    models.py                Pydantic request/response models
  xr/                      Platform-agnostic XR layer
    types.py                 XR message types and enums
    protocol.py              XR protocol codec
    handler.py               Universal XR WebSocket handler
    session.py               XR session manager
    adapters/spectacles.py   Snap Spectacles adapter
  networking/              Transport layer
    collar.py                ESP32-S3 collar WebSocket handler
    glasses.py               Snap Spectacles WebSocket handler
    supabase_rt.py           Supabase Realtime bridge (Spectacles broadcast)
    simulator.py             Software device simulator
  hdl/                     Hardware Description Language
    grammar.py               Five-dimensional wearable grammar
    codesign.py              AI-assisted hardware co-design
    library/devices.py       Pre-built device specs (collar, spectacles)

server/                    Thin FastAPI shell
  app/main.py                FastAPI app with lifespan management
  app/routers/               HTTP/WS route handlers (ai, collar, hdl, system)
  static/dashboard.html      Live OS dashboard
  tests/                     Integration tests

collar/                    ESP32-S3 Arduino firmware
  firmware/contact_mic.ino   Piezo sensor, haptics, WebSocket client
  WIRING.md                  Hardware wiring guide

scripts/
  demo.sh                   Launch server + 2 simulators + dashboard
  tunnel.sh                 ngrok tunnel for public access

docs/
  spectacles-protocol.json  XR protocol spec (all message types)

• OS has zero dependencies on networking, AI, or FastAPI
• Runtime depends only on OS
• Networking depends on OS + AI (for signal processing)
• AI depends on OS types only (for models)
• XR depends on OS types only
• Server depends on all layers but is a thin orchestration shell

Device -> Bus -> StateEstimator -> PolicyManager -> OutputRouter -> Device
                                       |
                              LLM Mediator (complex situations)

# Install the OS package (from project root)
uv venv && uv pip install -e ".[dev]"

# Set up the server
cd server
cp .env.example .env    # fill in API keys (see below)
uv venv && uv pip install -e ".[dev]" && uv pip install -e ..

# Run
uv run uvicorn app.main:app --host 0.0.0.0 --port 8000

Or use the demo script to start the server with simulated users:

bash scripts/demo.sh

Dashboard: http://localhost:8000/dashboard API docs: http://localhost:8000/docs Public access (hackathon): ngrok http 8000

Create server/.env from .env.example:

The Supabase bridge connects DelightfulOS to Snap Spectacles via broadcast channels on the spectacles channel (no database writes needed — pure WebSocket pub/sub).

Spectacles -> Server:

Server -> Spectacles:

Bidirectional real-time audio via Google's native audio API. The collar streams 16kHz PCM audio to the server, which pipes it to Gemini Live and streams 24kHz audio responses back.

• Input/output transcription via Gemini's built-in transcription
• Context window compression for unlimited session duration
• Session resumption across reconnects
• Artifact generation (summaries, meeting notes, action items) from accumulated transcripts
• Transcriptions are automatically forwarded to Spectacles via the Supabase bridge

The XR WebSocket protocol (/system/xr/ws/{user_id}) is platform-agnostic. See docs/spectacles-protocol.json for the full spec.

Client -> Server: scene_update, gesture, pinch, gaze_shift, voice_command, heartbeat Server -> Client: show_overlay, remove_overlay, highlight_user, toast, social_cue, haptic, mode_change, state_update

Supported platforms: Spectacles (active), Quest/Vision Pro/WebXR (protocol-ready).

cd server

# Unit tests (no API keys needed)
uv run python -m tests.test_signal

# API integration tests (needs .env with PRIME_API_KEY)
uv run python -m tests.test_prime_api

# Server integration tests (needs running server)
uv run python -m tests.test_server

The collar is an ESP32-S3 with a piezo contact microphone and haptic motor. See collar/WIRING.md for the wiring guide and collar/firmware/contact_mic.ino for the Arduino firmware.

• Server: Python 3.12+, FastAPI, uvicorn, pydantic
• AI inference: Prime Intellect (OpenAI-compatible, 100+ models), Google Gemini Live
• XR transport: Supabase Realtime (Phoenix WebSocket protocol via snapcloud.dev)
• XR client: Snap Spectacles via Lens Studio
• Hardware: ESP32-S3, piezo contact mic, haptic motor
• Package management: uv