Overview

Relevant source files

• README.md
• app/lib/backend/preferences.dart
• app/lib/main.dart
• app/lib/pages/home/page.dart
• app/lib/pages/home/widgets/battery_info_widget.dart
• app/lib/pages/memories/page.dart
• app/lib/pages/settings/developer.dart
• app/lib/pages/settings/privacy.dart
• app/lib/providers/capture_provider.dart
• app/lib/providers/developer_mode_provider.dart
• app/lib/providers/home_provider.dart
• app/lib/services/wals.dart
• app/lib/utils/other/time_utils.dart
• backend/database/apps.py
• backend/database/chat.py
• backend/database/memories.py
• backend/database/redis_db.py
• backend/database/vector_db.py
• backend/main.py
• backend/migrations/005_backfill_memory_vectors.py
• backend/modal/speech_profile_modal.py
• backend/models/chat.py
• backend/models/transcript_segment.py
• backend/requirements.txt
• backend/routers/apps.py
• backend/routers/chat.py
• backend/routers/custom_auth.py
• backend/routers/mcp_sse.py
• backend/routers/memories.py
• backend/routers/plugins.py
• backend/routers/pusher.py
• backend/routers/speech_profile.py
• backend/routers/sync.py
• backend/routers/transcribe.py
• backend/scripts/stt/j_apply_vad_to_speech_profiles.py
• backend/scripts/stt/k_compare_transcripts_performance.py
• backend/scripts/users/memories_fixes.py
• backend/scripts/users/retrieval.py
• backend/tests/unit/test_pusher_heartbeat.py
• backend/tests/unit/test_transcript_segment.py
• backend/utils/apps.py
• backend/utils/other/storage.py
• backend/utils/pusher.py
• backend/utils/retrieval/graph.py
• backend/utils/stt/speech_profile.py
• backend/utils/stt/streaming.py
• backend/utils/stt/vad.py
• backend/utils/webhooks.py
• omi/firmware/omi/README.md
• omi/firmware/readme.md
• omi/firmware/scripts/docker-build.md

Purpose and Scope

Omi is an open-source AI wearable platform that captures conversations, transcribes them in real-time, and extracts structured insights (conversations/memories). The repository contains four main subsystems:

Component	Location	Technology	Primary Entry Point
Backend	backend/	Python FastAPI	backend/main.py
Mobile App	app/	Flutter (Dart)	app/lib/main.dart
Desktop App	Friend/	Swift (macOS)	Friend/Friend/FriendApp.swift
Firmware	omi/firmware/	Zephyr RTOS (C/C++)	omi/firmware/omi/src/main.c

This overview summarizes the high-level architecture, data flow, and key integration points. For detailed documentation:

• Backend services and APIs: 3
• Mobile application architecture: 4
• Desktop application: 5
• Hardware and firmware: 6
• Plugin development: 7

Sources:

• README.md67-74
• backend/main.py1-141
• app/lib/main.dart1-513

---

System Components

The Omi platform is organized into four primary subsystems, each with distinct responsibilities and technology stacks.

Component Structure

Diagram: Repository Structure and Communication Flow

Sources:

• backend/main.py62-104
• backend/routers/transcribe.py121-136
• backend/routers/pusher.py1-300
• app/lib/main.dart120-214
• app/lib/providers/capture_provider.dart62-189
• app/lib/pages/home/page.dart1-100

Technology Stack Overview

Component	Primary Language	Key Frameworks	Purpose
Firmware	C/C++	Zephyr RTOS 2.9.0	Audio capture, OPUS encoding, BLE streaming, SD card storage
Mobile App	Dart	Flutter 3.x, Provider	Cross-platform UI, BLE connection, audio capture, state management
Desktop App	Swift	SwiftUI, Combine	macOS screen recording, file indexing, agent bridge
Backend	Python 3.11+	FastAPI, LangChain, LangGraph	Real-time transcription, conversation processing, AI chat, vector search

Sources:

• README.md67-74
• omi/firmware/readme.md1-74
• app/lib/main.dart1-513

---

Core User Flow

The typical user interaction follows this path: (1) wearable device captures audio, (2) mobile app streams audio to backend, (3) backend transcribes and processes conversation, (4) structured memories are created and stored.

End-to-End Data Flow

Diagram: Audio Capture to Memory Storage Pipeline

Sources:

• backend/routers/transcribe.py121-476
• backend/routers/pusher.py1-50
• backend/utils/conversations/process_conversation.py (referenced)
• app/lib/providers/capture_provider.dart626-674
• backend/routers/conversations.py1-100

State Management Flow

The mobile app uses Flutter's Provider pattern with ChangeNotifier for reactive state. Key providers coordinate device connection, audio streaming, conversation storage, and chat state.

Diagram: Provider Dependency Graph and Notification Flow

Sources:

• app/lib/main.dart285-355
• app/lib/providers/capture_provider.dart62-476
• app/lib/pages/home/page.dart104-164

---

Data Storage Architecture

Omi employs a multi-tier storage strategy optimizing for different access patterns: Firestore for structured documents, Pinecone for vector semantic search, Redis for caching, and Google Cloud Storage for binary data.

Storage Layer Mapping

Diagram: Multi-tier Storage Architecture with Code References

Sources:

• backend/database/conversations.py (referenced)
• backend/database/memories.py1-100
• backend/database/chat.py1-150
• backend/database/vector_db.py1-200
• backend/database/redis_db.py32-56
• backend/utils/other/storage.py1-50

Cache Strategy

The system implements multi-tier caching to minimize database queries and improve response times.

Backend Cache Hierarchy:

Cache Tier	Implementation	TTL	Key Functions	Use Case
In-Memory	database/cache.py MemoryCache	Request lifetime	get_or_fetch() with singleflight	Approved apps, popular apps
Redis	database/redis_db.py	10-30 minutes	get_generic_cache() set_generic_cache()	App metadata, reviews, install counts
Firestore Cache	Backend persistent	Permanent	get_app_cache_by_id()	App definitions, user data

Mobile Cache:

Storage Type	Implementation	Persistence	Key Properties
SharedPreferences	SharedPreferencesUtil	Permanent	cachedConversations cachedPeople customSttConfig
WAL (Write-Ahead Log)	services/wals/	Until synced	WalService LocalWalSync

Sources:

• backend/database/redis_db.py32-56
• backend/utils/apps.py143-199
• app/lib/backend/preferences.dart16-100
• app/lib/services/wals.dart1-25

---

Key Service Integrations

Speech-to-Text Pipeline

The backend supports multiple STT providers with automatic selection based on language and user configuration. The mobile app can override the default provider via CustomSttConfig.

Diagram: STT Provider Selection and Processing

Supported Providers:

Provider	Model	Languages	Speech Profile Support	Selection Criteria
Deepgram Nova-3	nova-3	50+ multi-language	Yes (via include_speech_profile)	Default for most languages
Soniox	soniox_streaming	80+ including multi	Yes (via uid param)	Best for English, multi-language detection
Speechmatics	speechmatics_streaming	30+	Partial	Alternative for certain languages
Custom (Groq/OpenAI)	User-configured	Depends on model	No	User preference via CustomSttConfig

Sources:

• backend/routers/transcribe.py200-206
• backend/utils/stt/streaming.py26-196
• app/lib/backend/preferences.dart95-128
• app/lib/providers/capture_provider.dart440-463

Vector Search Architecture

Pinecone provides semantic search capabilities with OpenAI embeddings (text-embedding-3-large, 3072 dimensions). The system uses three namespaces to segregate different data types.

Diagram: Vector Storage and Retrieval Pipeline

Key Vector Operations:

Function	File	Purpose	Metadata Filters
upsert_conversation_vector()	database/vector_db.py	Index conversation transcript	user_id, created_at
upsert_memory_vector()	database/vector_db.py	Index memory with category	user_id, category, created_at
query_vectors()	database/vector_db.py	Semantic search with filters	Date range, category, top_k
query_vectors_by_metadata()	database/vector_db.py	Metadata-only queries	Specific date/category filters

Sources:

• backend/database/vector_db.py1-200
• backend/routers/mcp_sse.py160-181
• backend/utils/retrieval/graph.py1-100

---

Authentication and User Management

Authentication Flow

Firebase Authentication provides token-based authorization. The mobile app uses AuthService to obtain ID tokens, which the backend validates using Firebase Admin SDK.

Diagram: Authentication Token Validation Flow

Authentication State Management:

Component	Location	Persistence	Key Methods
Firebase Auth	firebase_admin.auth	Server-side	verify_id_token() get_user()
AuthService	services/auth_service.dart	Singleton	getIdToken() restoreOnboardingState()
SharedPreferences	SharedPreferencesUtil	Local persistent	uid onboardingCompleted

Sources:

• app/lib/main.dart175-186
• app/lib/backend/preferences.dart33-36
• app/lib/backend/preferences.dart286-288

---

Development and Deployment

Mobile App Build Pipeline

The Flutter app uses Codemagic for CI/CD with automatic deployments to TestFlight (iOS) and Google Play Internal Track (Android). Version validation prevents duplicate uploads.

Build Triggers:

• Auto-deploy: Push to main branch → Internal testing
• Production: Git tags matching v*-mobile-cm → TestFlight/Play Store

Source:

• README.md37-54

Backend Deployment

The Python backend runs on Google Cloud Run with automatic scaling. The FastAPI application is defined in backend/main.py and includes middleware for CORS, error handling, and request logging.

Key Configuration:

• Environment variables managed via backend/env/env.py and backend/env/prod_env.py
• Firebase Admin SDK for authentication and Firestore access
• Pinecone client for vector operations
• Redis connection for caching layer

Sources:

• app/lib/env/env.dart1-30 (mobile environment config reference)
• app/lib/env/prod_env.py (referenced)

Firmware Build System

The Zephyr firmware uses Docker-based builds for consistency across platforms. The build script scripts/build-docker.sh creates a containerized environment with Nordic NCS 2.9.0 toolchain.

Build Outputs:

• zephyr.uf2: Direct USB flashing to XIAO board
• zephyr.zip: OTA update package for mobile app
• zephyr.hex, zephyr.bin: Raw firmware images

Sources:

• omi/firmware/omi/README.md1-70
• omi/firmware/scripts/docker-build.md1-146

---

Plugin and Integration Ecosystem

The app marketplace supports multiple plugin types with different integration patterns. Plugins are defined in community-plugins.json and stored in Firestore's plugins_data collection.

Plugin Types and Capabilities

Apps extend Omi functionality through multiple integration patterns. The App model defines capabilities via capabilities list and integration config in external_integration.

Diagram: Plugin System Architecture

Plugin Capability Matrix:

Capability	Trigger Event	Processing Location	Key Fields	Example
memories	memory_created	routers/conversations.py	memory_prompt	Journaling, wellness tracking
chat	User message	routers/chat.py	chat_prompt	AI assistants, Q&A
external_integration	memory_created, transcript_processed, audio_bytes	utils/app_integrations.py	webhook_url, triggers_on	Zapier, Notion, custom webhooks
Persona	memory_created, chat	utils/llm/persona.py	persona_prompt, persona_description	Coach, mentor emulation

Sources:

• backend/models/app.py (referenced)
• backend/routers/apps.py200-434
• backend/routers/conversations.py (referenced)
• backend/utils/apps.py1-100

MCP (Model Context Protocol) Integration

The backend provides an MCP server for desktop agent integration. The server exposes tools for memory/conversation management and semantic search.

Diagram: MCP Server Architecture and Tool Execution

MCP Tool Reference:

Tool Name	Function	Arguments	Return Type	Database Operation
get_memories	execute_tool() line 176	limit, offset, categories	List[dict]	memories_db.get_memories()
create_memory	execute_tool() line 188	content, category	dict	memories_db.create_memory()
delete_memory	execute_tool() line 202	memory_id	dict	memories_db.delete_memory()
get_conversations	execute_tool() line 219	limit, offset	List[dict]	conversations_db.get_conversations()
semantic_search	execute_tool() line 232	query, limit	List[dict]	vector_db.query_vectors()

Sources:

• backend/routers/mcp_sse.py160-262
• app/lib/pages/settings/developer.dart230-286
• backend/database/memories.py30-65
• backend/database/conversations.py (referenced)

---

Summary

The Omi platform integrates four major subsystems—wearable firmware, mobile app, desktop app, and backend services—to create a seamless AI wearable experience. Audio flows from hardware devices through BLE or system capture to the FastAPI backend, where it's transcribed in real-time, processed into structured conversations and memories, and stored across Firestore, Pinecone, and Redis. The plugin ecosystem extends functionality through memory apps, chat apps, personas, and MCP integrations, while the multi-platform architecture (iOS, Android, macOS, Zephyr) ensures broad device compatibility.

For in-depth information on specific components, refer to the subsystem documentation pages linked at the beginning of this overview.