CoreDex is an offline, on-device multimodal AI assistant for iOS that detects real-world objects in real time and provides intelligent explanations through voice interaction — all without requiring an internet connection.
CoreDex integrates computer vision, on-device language reasoning, and speech interaction into a single iOS application.
The system runs entirely on-device, ensuring low latency, privacy, and reliability even in zero-connectivity environments.
The project is designed as a base framework that can be extended with custom models for different domains.
- Real-time Object Detection using YOLO-based Core ML models
- On-device Reasoning for object understanding and explanations
- Voice Interaction with speech-to-text and text-to-speech
- Fully Offline Execution (no network dependency)
- Modular Architecture supporting custom model uploads
- Hardware-Accelerated Inference via Apple Neural Engine
- Live camera feed is captured from the device
- Objects are detected in real time using a Core ML model
- Detected object labels and confidence are passed to the reasoning module
- On-device language model generates contextual explanations
- Voice output is delivered using text-to-speech
- Swift
- SwiftUI
- Core ML
- YOLO (Core ML–converted models)
- Apple Neural Engine (A-/M-series chips)
- AVFoundation (camera & audio handling)
- Speech Framework (speech-to-text)
- AVSpeechSynthesizer (text-to-speech)
- On-device Foundation Models
- On-device custom model training
- Direct visual context integration for deeper reasoning
- Hybrid online + offline support for advanced reasoning
- Expanded domain-specific detection models