A complete implementation of a 2D differential-drive robot simulation featuring a closed-loop perception and control system. This project demonstrates autonomous navigation, target detection, and state-machine-based control strategies.

• Overview
• Features
• Demo
• System Architecture
• Installation
• Usage
  • Quick Start
  • Running Modes
  • Command-Line Options
• Project Structure
• Configuration
• Testing and Evaluation
• Technical Details
• Future Enhancements
• License

The robot autonomously navigates to sequential targets using vision-based perception and finite-state machine control.

This project implements a virtual 2D autonomous robot that demonstrates the complete perception-control-action pipeline used in real robotics systems. The robot navigates through a simulated environment to sequentially reach multiple targets while avoiding distractors and obstacles.

Pipeline: Sense -> Perceive -> Control -> Act

The system simulates realistic challenges including:

• Sensor noise and measurement uncertainty
• Perception latency
• Limited field-of-view constraints
• Dynamic decision-making under uncertainty

• Geometry-based Robot-Centric Camera: Projects targets, distractors, and obstacles within a limited FOV for realistic perception simulation
• Computer Vision Perception: HSV color thresholding with morphological operations, contour filtering, and confidence scoring
• State Machine Control: Three-state FSM (SEARCH/TRACK/APPROACH) with adaptive speed scheduling
• Differential Drive Simulation: Realistic kinematics for a two-wheeled robot with configurable constraints
• Robustness Features: Configurable perception latency, measurement noise, and exponential smoothing

• Sequential target navigation (targets disappear upon reaching)
• Optional cyan distractors to test robustness
• Static obstacles for collision detection scenarios
• High-resolution visualization (1920x1080)

• Per-step CSV telemetry logging (pose, commands, detections, errors, control mode)
• GIF capture for visualizations
• Batch evaluation with performance metrics export
• Debug overlay showing camera FOV, detection masks, and state information

+-------------+    +--------------+    +-------------+    +----------+
|   Camera    |--->|  Perception  |--->|   Control   |--->|  Robot   |
|  (Sensing)  |    |   (Vision)   |    |    (FSM)    |    | (Action) |
+-------------+    +--------------+    +-------------+    +----------+
      |                    |                   |                  |
      |                    |                   |                  |
      +--------------------+-------------------+------------------+
                                  |
                          +-------v--------+
                          |  World State   |
                          |  (Simulation)  |
                          +----------------+

1. Simulation (src/sim.py): World state management, robot kinematics, target/obstacle handling
2. Perception (src/perception.py): Color-based target detection with OpenCV
3. Control (src/control.py): Finite-state machine with heading control
4. Estimation (src/estimation.py): Sensor fusion and smoothing algorithms
5. Rendering (src/render.py): Pygame-based visualization with debug overlays

Run the default demo with distractors and obstacles:

python src/run_demo.py

python src/run_demo.py --steps 2000 --seed 0 --no-distractors --no-obstacles --camera-mode robot --debug-overlay

python src/run_demo.py --clean-gif

python src/run_demo.py --headless --steps 600 --log-csv outputs/run1.csv --save-gif --gif-path outputs/run1.gif

python src/run_demo.py --headless --steps 600 --perception-latency 3 --meas-noise-px 2.0 --smooth-alpha 0.4 --log-csv outputs/run_latency.csv

python src/eval.py --episodes 10 --steps 600 --camera-mode robot --no-distractors --no-obstacles --metrics-csv outputs/metrics.csv

robotics-2d-perception-control/
├── assets/                    # Demo GIFs and visual assets
│   └── clean-run.gif
├── outputs/                   # Generated logs and visualizations
│   ├── logs/
│   └── sanity_logs/
├── src/                       # Source code
│   ├── __init__.py
│   ├── config.py             # Simulation configuration parameters
│   ├── control.py            # FSM controller and heading control
│   ├── estimation.py         # Smoothing and filtering algorithms
│   ├── eval.py               # Batch evaluation script
│   ├── perception.py         # Vision-based target detection
│   ├── render.py             # Pygame visualization
│   ├── run_demo.py           # Main demo entry point
│   └── sim.py                # Physics and world simulation
├── requirements.txt          # Python dependencies
├── LICENSE                   # MIT License
└── README.md                 # This file

SimConfig:
    # World
    width: 1920 px
    height: 1080 px
    dt: 1/60 s (60 Hz simulation)
    
    # Robot
    wheel_base: 40.0 px
    v_max: 140.0 px/s (linear velocity limit)
    w_max: 3.2 rad/s (angular velocity limit)
    
    # Camera
    cam_w: 160 px
    cam_h: 120 px
    cam_fov: 1.2 rad (~69 degrees)
    
    # Target
    target_radius: 16 px

python src/eval.py --episodes 20 --steps 800 --metrics-csv outputs/metrics.csv

step, sim_time_s, robot_x, robot_y, robot_theta, target_x, target_y, distance_to_target,
v_cmd, w_cmd, detected, detected_cx, detected_cy, detect_conf, err_norm, err_norm_filtered,
mode, holding_estimate

import pandas as pd
df = pd.read_csv('outputs/run1.csv')
# Analyze trajectory, control behavior, detection performance

• Replace color thresholding with learned CNN detector
• Implement obstacle avoidance using reactive control or path planning
• Add trajectory tracking task with waypoint following
• Integrate SLAM for environment mapping
• Support for dynamic obstacles and multi-robot scenarios
• ROS2 integration for hardware deployment

This project is licensed under the MIT License - see the LICENSE file for details.