A high-performance, privacy-first web application for real-time object detection that runs entirely in your browser using WebGPU acceleration and ONNXRuntime-Web. No server required, no data leaves your device.

• 🚀 High-Speed Inference: 30-60 FPS on laptops, 15-30 FPS on modern mobile devices
• 🔒 Complete Privacy: All processing happens locally, zero data transmission
• ⚡ WebGPU Accelerated: Leverages GPU for maximum performance
• 📱 Mobile Compatible: Works on both desktop and mobile browsers
• 🎥 Multi-Camera Support: Switch between front/back cameras
• 🎨 Real-Time Visualization: Live bounding boxes with class labels and confidence scores
• 📊 Performance Monitoring: Real-time FPS counter and execution provider display
• 🌐 Offline Capable: Works without internet after initial load

• Frontend Framework: React 18.2 with Hooks
• Build Tool: Vite 5.0 (fast HMR and optimized builds)
• AI Runtime: ONNX Runtime Web 1.17
• Execution Provider: WebGPU (with WebAssembly fallback)
• Model: YOLOv8n (80 COCO classes)

realTimeOD/
├── public/
│   └── models/
│       └── yolov8n.onnx          # Place your ONNX model here
├── src/
│   ├── components/
│   │   ├── App.jsx               # Main application component
│   │   ├── CameraView.jsx        # Video display with overlays
│   │   ├── BoundingBoxCanvas.jsx # Detection visualization
│   │   ├── Controls.jsx          # UI controls
│   │   └── FPSMonitor.jsx        # Performance metrics
│   ├── hooks/
│   │   ├── useCamera.js          # Camera management logic
│   │   └── useObjectDetection.js # Detection pipeline logic
│   ├── utils/
│   │   ├── constants.js          # Configuration and constants
│   │   ├── modelLoader.js        # ONNX model loading
│   │   ├── preprocessing.js      # Image preprocessing
│   │   └── postprocessing.js     # NMS and output processing
│   ├── main.jsx                  # Application entry point
│   └── index.css                 # Global styles
├── index.html
├── package.json
├── vite.config.js
└── README.md

• Node.js 18+ installed
• Modern browser with WebGPU support (Chrome 113+, Edge 113+)
• YOLOv8n ONNX model file

1. Clone or navigate to the project directory:

   cd realTimeOD

2. Install dependencies:

   npm install

3. Download the YOLOv8n ONNX model:

   You can export YOLOv8n to ONNX format using the Ultralytics library:

   pip install ultralytics

   from ultralytics import YOLO
   
   # Load YOLOv8n model
   model = YOLO('yolov8n.pt')
   
   # Export to ONNX format
   model.export(format='onnx', opset=12, simplify=True, dynamic=False, imgsz=640)

   Then move the generated yolov8n.onnx file to public/models/:

   mkdir -p public/models
   mv yolov8n.onnx public/models/

4. Start the development server:

   npm run dev

5. Open your browser and navigate to http://localhost:3000

1. Allow camera access when prompted
2. Click "Load Model" to initialize the ONNX model
3. Click "Start Detection" to begin real-time object detection
4. Use the camera selector to switch between devices
5. Monitor FPS and detection count in the sidebar

Edit src/utils/constants.js to adjust detection parameters:

export const MODEL_CONFIG = {
  modelPath: "/models/yolov8n.onnx",
  inputWidth: 640,
  inputHeight: 640,
  confidenceThreshold: 0.25, // Minimum confidence to show detections
  iouThreshold: 0.45, // IoU threshold for NMS
  maxDetections: 100, // Maximum number of detections
};

Adjust camera settings in src/utils/constants.js:

export const CAMERA_CONFIG = {
  defaultWidth: 640,
  defaultHeight: 480,
  idealWidth: 1280,
  idealHeight: 720,
  facingMode: "user", // 'user' for front, 'environment' for back
};

1. Build the application:

   npm run build

2. Preview the production build:

   npm run preview

The optimized build will be in the dist/ directory, ready for deployment.

Deploy to any static hosting service:

• Netlify: Drag and drop dist/ folder or connect Git repo
• Vercel: vercel deploy
• GitHub Pages: Push dist/ to gh-pages branch
• Cloudflare Pages: Connect repo and deploy

1. HTTPS Required: Camera access requires HTTPS in production
2. Headers for ONNX Runtime: Some hosts may need CORS headers configured
3. Model File: Ensure yolov8n.onnx is included in deployment

Create netlify.toml:

[build]
  command = "npm run build"
  publish = "dist"

[[headers]]
  for = "/*"
  [headers.values]
    Cross-Origin-Embedder-Policy = "require-corp"
    Cross-Origin-Opener-Policy = "same-origin"

To use a different YOLO model:

1. Export your model to ONNX format with input size 640x640
2. Place it in public/models/
3. Update modelPath in src/utils/constants.js
4. Adjust output processing if using different YOLO version

• Ensure you're using HTTPS (or localhost)
• Check browser permissions for camera access
• Try a different browser

• Verify yolov8n.onnx exists in public/models/
• Check browser console for error messages
• Ensure model was exported correctly

• Check if WebGPU is enabled (see execution provider in UI)
• Try reducing camera resolution
• Close other GPU-intensive applications
• Consider using a lighter model (YOLOv8n is already the smallest)

The app automatically falls back to WebAssembly. Performance will be lower but still functional.

To enable WebGPU:

• Chrome/Edge: Should be enabled by default in version 113+
• Firefox: Not yet supported (use WASM fallback)
• Safari: Limited support, may need enabling in settings

Edit src/utils/constants.js to modify the class list:

export const COCO_CLASSES = [
  "person",
  "bicycle",
  "car", // ... your classes
];

Modify the color palette in src/utils/constants.js:

export const COLORS = [
  "#FF6B6B",
  "#4ECDC4", // ... your colors
];

All component styles are in separate CSS files:

• src/App.css - Main layout
• src/components/*.css - Component-specific styles

1. Reduce Input Resolution: Lower resolution = faster inference
2. Adjust Confidence Threshold: Higher threshold = fewer detections to process
3. Use OffscreenCanvas: Automatically used when available
4. Optimize Camera Settings: Lower camera resolution if not needed
5. Close Unnecessary Tabs: Free up GPU resources

• No Data Collection: Nothing is sent to any server
• Local Processing: All inference happens on your device
• No Cookies: No tracking or analytics
• Open Source: Fully inspectable code

Apache 2.0 License - feel free to use this project for any purpose.

• Ultralytics YOLOv8 - Object detection model
• ONNX Runtime Web - Inference engine
• React - UI framework
• Vite - Build tool

For issues, questions, or contributions:

• Open an issue on GitHub
• Check existing documentation
• Review troubleshooting section

Potential improvements:

• Multiple model support (YOLOv8s, YOLOv8m, etc.)
• Video file upload support
• Screenshot/recording functionality
• Detection statistics and analytics
• Custom model training integration
• Pose estimation support
• Segmentation support

Built with ❤️ for privacy-conscious AI applications