A small, reinforcement learning project: train PPO on CartPole, evaluate with baselines + confidence intervals, and keep runs reproducible (CI included).

• Demo
• Results (example)
• Skills / signals
• Tech stack
• Quickstart
• Reproducibility
• Repo structure
• Future plans

Generated via multi-seed evaluation (20 episodes/seed × 3 seeds) and exported to CSV/Markdown.

Artifacts: outputs/comparisons/results.md, outputs/comparisons/results.csv

• RL training loop + evaluation discipline (baselines, multi-seed, confidence intervals)
• Reproducibility (configs saved per run, deterministic seeds)
• ML/SWE hygiene (CI, tests, formatting, artifacts)

• Gymnasium (classic-control), Stable-Baselines3 (PPO), PyTorch
• Experiment artifacts: JSON metrics + TensorBoard
• Evaluation: baselines + multi-seed rollouts + 95% confidence intervals

• Python 3.8+
• FFmpeg (required only for video/GIF recording via moviepy/imageio[ffmpeg])

python -m venv .venv
# Windows: .venv\Scripts\activate
# macOS/Linux: source .venv/bin/activate

# Install (matches CI)
python -m pip install --upgrade pip
pip install -e ".[dev]"

# 1) Train (writes outputs/runs/<timestamp>_<env>_seed<seed>_<git-hash>/)
python -m src.train --config configs/default.yaml

# 2) Compare vs baselines (writes outputs/comparisons/results.*)
python -m src.compare --model-path outputs/runs/<run-dir>/model.zip --episodes 20 --num-seeds 3 --seed 123

# 3) View training curves in TensorBoard
tensorboard --logdir outputs/runs/<run-dir>/tensorboard

# 4) Record a short demo GIF (writes videos/ and/or assets/ depending on your config)
python -m src.record_video --model-path outputs/runs/<run-dir>/model.zip --gif

Optional: Multi-seed evaluation with saved results

python -m src.evaluate --model-path outputs/runs/<run-dir>/model.zip --episodes 20 --num-seeds 5 --save-results

• Re-run an existing experiment from its saved config:

python -m src.reproduce outputs/runs/<run-dir>

• Validate determinism and core behaviors via tests:

pytest -q

• src/ - core scripts: train, evaluate, compare, reproduce, record_video, plus shared utilities in common
• configs/ - YAML configs (default.yaml, quick_test.yaml)
• outputs/runs/ - per-run artifacts (config, model, metrics, TensorBoard logs)
• outputs/comparisons/ - exported results tables (CSV/Markdown)
• assets/, videos/ - demo media

pip install -e ".[dev]"

rl-train --config configs/default.yaml
rl-evaluate --model-path outputs/runs/<run-dir>/model.zip
rl-compare --model-path outputs/runs/<run-dir>/model.zip
rl-reproduce outputs/runs/<run-dir>
rl-video --model-path outputs/runs/<run-dir>/model.zip

# Quality
make lint
make format
make test

# Training / evaluation
make train-quick
make evaluate
make compare

• Add a second environment (Acrobot-v1), compare training curves
• Add hyperparameter sweep with results table