Website  ·  Docs  ·  Dataset  ·  Model  ·  NAVSIM Model  ·  Supplementary  ·  Paper

An open-source end-to-end driving stack for CARLA.
▶ State-of-the-art performance on all major Leaderboard 2.0 benchmarks ◀

Long Nguyen^1,3  ·   Micha Fauth¹  ·   Bernhard Jaeger^1,3  ·   Daniel Dauner^1,3
Maximilian Igl²  ·   Andreas Geiger^1,3  ·   Kashyap Chitta²

¹University of Tübingen, Tübingen AI Center  ·  ²NVIDIA Research  ·  ³KE:SAI

• Table of Contents
• Updates
• 1. Quick Start
  • 1.1. Environment initialization
  • 1.2. Install dependencies
  • 1.3. Download checkpoints
  • 1.4. Setup VSCode/PyCharm
  • 1.5. Evaluate model
  • 1.6. Infraction Analysis Webapp
• 2. CARLA Research Cycle
  • 2.1. (Optional) Extending Data Routes
  • 2.2. Collecting Data
  • 2.3. Training
  • 2.4. Benchmarking
• 3. Extensions
  • 3.1. Fail2Drive Evaluation
  • 3.2. CaRL Agent Evaluation
  • 3.3. NAVSIM Training and Evaluation
  • 3.4. CARLA 123D Data Collection
• 4. Project Structure
• 5. Common Issues
• Beyond CARLA: Cross-Benchmark Deployment
• Further Documentation
• Acknowledgements
• Citation

Get LEAD running locally: from cloning the repo and installing dependencies, to downloading a pretrained checkpoint and driving a CARLA Leaderboard 2.0 route end-to-end. We tested the instructions on the following configurations:

Clone the repository and register the project root:

git clone https://github.com/kesai-labs/lead.git
cd lead

# Set project root variable
echo -e "export LEAD_PROJECT_ROOT=$(pwd)" >> ~/.bashrc

# Activate project's hook
echo "source $(pwd)/scripts/main.sh" >> ~/.bashrc

# Reload shell config
source ~/.bashrc

Verify that ~/.bashrc reflects these paths correctly.

We use Miniconda as container and uv for Python dependencies. Runtime and dev dependencies are declared entirely in pyproject.toml.

# (Optional, needed in some cases) Accept terms and services
conda tos accept --override-channels --channel \
  https://repo.anaconda.com/pkgs/main
conda tos accept --override-channels --channel \
  https://repo.anaconda.com/pkgs/r

# Create a conda environment
conda create -n lead python=3.10 -y
conda activate lead

# Install system-level tools
conda install -c conda-forge \
  ffmpeg parallel tree gcc zip unzip git-lfs uv

# Tell uv to use conda environment
mkdir -p $CONDA_PREFIX/etc/conda/activate.d \
  $CONDA_PREFIX/etc/conda/deactivate.d
echo 'export VIRTUAL_ENV=$CONDA_PREFIX' > \
  $CONDA_PREFIX/etc/conda/activate.d/uv.sh
echo 'unset VIRTUAL_ENV' > \
  $CONDA_PREFIX/etc/conda/deactivate.d/uv.sh

# Reactivate conda environment
conda activate lead

# Install dependencies
uv sync --active --extra dev

# Optional: activate git hooks
pre-commit install

Set up CARLA:

# Download and setup CARLA at 3rd_party/CARLA_0915
bash scripts/setup_carla.sh

# Or symlink your pre-installed CARLA
ln -s /your/carla/path 3rd_party/CARLA_0915

# Add dependency
uv add --active <pkg>

# Add dev dependency
uv add --active --optional dev <pkg>

# Update uv.lock
uv lock

Pre-trained checkpoints are hosted on HuggingFace. Following are the results from the paper. To reproduce these results, enable the Kalman filter, stop-sign, and creeping heuristics by setting sensor_agent_creeping=True use_kalman_filter=True slower_for_stop_sign=True in config_closed_loop.

Without these heuristics, the performance changes as follows, with the biggest boost observed on Town13.

Download the checkpoints:

# Download one checkpoint for testing
bash scripts/download_one_checkpoint.sh

# Download all checkpoints
git clone https://huggingface.co/ln2697/tfv6 outputs/checkpoints
cd outputs/checkpoints
git lfs pull

VSCode	PyCharm
Install the recommended extensions when prompted. Debugging works out of the box.	Add the CARLA Python API 3rd_party/CARLA_0915/PythonAPI/carla to your interpreter paths via Settings → Python → Interpreter → Show All → Show Interpreter Paths.

Verify your setup with a single route:

# Start driving environment
bash scripts/start_carla.sh

# Turn on images and videos output
export LEAD_CLOSED_LOOP_CONFIG="produce_demo_image=true \
  produce_demo_video=true \
  produce_debug_image=true \
  produce_debug_video=true \
  produce_input_image=true \
  produce_input_video"

# Run policy on one route
python lead/leaderboard_wrapper.py \
  --checkpoint outputs/checkpoints/tfv6_resnet34 \
  --routes data/benchmark_routes/bench2drive/23687.xml \
  --bench2drive

Driving logs are saved to outputs/local_evaluation/<route_id>/:

mv outputs/checkpoints/tfv6_resnet34/model_0030_1.pth \
outputs/checkpoints/tfv6_resnet34/_model_0030_1.pth

mv outputs/checkpoints/tfv6_resnet34/model_0030_2.pth \
outputs/checkpoints/tfv6_resnet34/_model_0030_2.pth

Launch the interactive infraction dashboard to analyze driving failures — especially useful for Longest6 or Town13 where iterating over evaluation logs is time-consuming:

python lead/infraction_webapp/app.py

Navigate to http://localhost:5000 and point it at outputs/local_evaluation.

The primary focus of this repository is solving the original CARLA Leaderboard 2.0. This section walks through the full research loop — collecting expert demonstrations, training a TFv6 policy, and benchmarking it closed-loop.

Before collecting, you may want to enlarge or diversify the route set under data/data_routes/. This step is optional — the shipped routes are enough to reproduce the paper results. However, if you want to improve the performance of the model, in particular for Longest6 v2 or Fail2Drive, introduce more routes is the easiest way to achieve this.

Two ways to do it:

• Automatic — sample routes programmatically from a CARLA town (e.g. random start/goal pairs with scenario annotations). Useful when you want large-scale coverage without hand-authoring.
• Manual — use the bundled carla_route_generator, a GUI tool for clicking waypoints on a map and exporting routes. Launch it via the hotkey script:

cd 3rd_party/carla_route_generator
python3 scripts/window.py

Generated XML files can be dropped directly into data/data_routes/ and picked up by the expert during data collection.

With CARLA running, collect data for a single route via Python (recommended for debugging):

python lead/leaderboard_wrapper.py \
  --expert \
  --routes data/data_routes/lead/noScenarios/short_route.xml

Or via bash (recommended for flexibility):

bash scripts/eval_expert.sh

Collected data is saved to outputs/expert_evaluation/ with the following structure:

On a SLURM Cluster of 92 GTX 1080ti, the data collection is often finished after 2 days.

Download the dataset from HuggingFace:

# Download all routes
git clone https://huggingface.co/datasets/ln2697/lead_carla data/carla_leaderboard2/zip
cd data/carla_leaderboard2/zip
git lfs pull

# Or download a single route for testing
bash scripts/download_one_route.sh

# Unzip the routes
bash scripts/unzip_routes.sh

# Build data cache
python scripts/build_cache.py

Perception pretraining. Logs and checkpoints are saved to outputs/local_training/pretrain:

# Single GPU
python3 lead/training/train.py \
  logdir=outputs/local_training/pretrain

# Distributed Data Parallel
bash scripts/pretrain_ddp.sh

Planning post-training. Logs and checkpoints are saved to outputs/local_training/posttrain:

# Single GPU
python3 lead/training/train.py \
  logdir=outputs/local_training/posttrain \
  load_file=outputs/local_training/pretrain/model_0030.pth \
  use_planning_decoder=true

# Distributed Data Parallel
bash scripts/posttrain_ddp.sh

With CARLA running, evaluate on any benchmark via Python:

python lead/leaderboard_wrapper.py \
  --checkpoint outputs/checkpoints/tfv6_resnet34 \
  --routes <ROUTE_FILE> \
  [--bench2drive]

Or via bash:

bash scripts/eval_bench2drive.sh   # Bench2Drive
bash scripts/eval_longest6.sh      # Longest6 v2
bash scripts/eval_town13.sh        # Town13
bash scripts/eval_fail2drive.sh    # Fail2Drive (requires CARLA_F2D)

Results are saved to outputs/local_evaluation/ with videos, infractions, and metrics.

Beyond the core Leaderboard 2.0 workflow, LEAD also supports additional benchmarks (Fail2Drive, NAVSIM), an alternative RL policy (CaRL), and an alternative data format (123D). Each extension plugs into the code base with minimal changes.

Fail2Drive is a CARLA Leaderboard 2 benchmark for testing closed-loop generalization on unseen long-tail scenarios.

Setup. Download the Fail2Drive simulator (custom CARLA build with novel assets):

mkdir -p 3rd_party/CARLA_F2D
curl -L https://hf.co/datasets/SimonGer/Fail2Drive/resolve/main/fail2drive_simulator.tar.gz \
  | tar -xz -C 3rd_party/CARLA_F2D

Evaluate. With CARLA_F2D running, evaluate on a single route:

# Start the Fail2Drive CARLA simulator
bash 3rd_party/CARLA_F2D/CarlaUE4.sh

# Evaluate model on one route
LEAD_CLOSED_LOOP_CONFIG="sensor_agent_creeping=True \
  use_kalman_filter=True \
  slower_for_stop_sign=True" \
python lead/leaderboard_wrapper.py \
  --checkpoint outputs/checkpoints/tfv6_regnety \
  --routes data/benchmark_routes/fail2drive/Base_Animals_0075.xml \
  --fail2drive

Results.

With CARLA running, evaluate the CaRL agent via Python:

CUBLAS_WORKSPACE_CONFIG=:4096:8 \
python lead/leaderboard_wrapper.py \
  --checkpoint outputs/checkpoints/CaRL \
  --routes data/benchmark_routes/bench2drive/24240.xml \
  --carl-agent \
  --bench2drive \
  --timeout 900

Or via bash:

bash scripts/eval_carl.sh

The results are in outputs/local_evaluation/<route_id>/.

Setup. Install navtrain and navtest splits following navsimv1.1/docs/install.md, then install the navhard split following navsimv2.2/docs/install.md.

Training. Run perception pretraining (script) followed by planning post-training (script). We use one seed for pretraining and three seeds for post-training to estimate performance variance.

Evaluation. Run evaluation on navtest and navhard.

With CARLA running, collect data in 123D format via Python:

export LEAD_EXPERT_CONFIG="target_dataset=6 \
  py123d_data_format=true \
  use_radars=false \
  lidar_stack_size=2 \
  save_only_non_ground_lidar=false \
  save_lidar_only_inside_bev=false"

python -u $LEAD_PROJECT_ROOT/lead/leaderboard_wrapper.py \
    --expert \
    --py123d \
    --routes data/data_routes/50x38_Town12/ParkingCrossingPedestrian/3250_1.xml

Or via bash:

bash scripts/eval_expert_123d.sh

Output in 123D format is saved to data/carla_leaderboard2_py123d/:

To visualize collected scenes in 3D with Viser:

python scripts/123d_viser.py

The project is organized into the following top-level directories. See the full documentation for a detailed breakdown.

If you face training instability, as reported in issue #67, try following solutions:

1. Turn off mixed-precision training in config.
2. Purge the Conda environment and reinstall from scratch.
3. Try the latest working commit at a41d11616.

The LEAD pipeline and TFv6 models serve as reference implementations across multiple E2E driving platforms:

For a deeper dive, visit the full documentation site:

Data Collection  ·  Training  ·  Evaluation.

The documentation will be updated regularly.

This project builds on the shoulders of excellent open-source work. Special thanks to carla_garage for the foundational codebase.

PDM-Lite  ·  Leaderboard  ·  Scenario Runner  ·  NAVSIM  ·  Waymo Open Dataset
SimLingo  ·  PlanT2  ·  Bench2Drive Leaderboard  ·  Bench2Drive  ·  CaRL  ·  Fail2Drive

Long Nguyen led development of the project. Kashyap Chitta, Bernhard Jaeger, and Andreas Geiger contributed through technical discussion and advisory feedback. Daniel Dauner provided guidance with NAVSIM.

If you find this work useful, please consider giving this repository a star and citing our paper:

@inproceedings{Nguyen2026CVPR,
	author = {Long Nguyen and Micha Fauth and Bernhard Jaeger and Daniel Dauner and Maximilian Igl and Andreas Geiger and Kashyap Chitta},
	title = {LEAD: Minimizing Learner-Expert Asymmetry in End-to-End Driving},
	booktitle = {Conference on Computer Vision and Pattern Recognition (CVPR)},
	year = {2026},
}