While end-to-end autonomous driving models show promising results, their practical deployment is often hindered by large model sizes, a reliance on expensive LiDAR sensors, and computationally intensive BEV feature representations. This limits their scalability, especially for mass-market vehicles equipped only with cameras. To address these challenges, we propose PRIX (Plan from Raw Pixels). Our novel and efficient end-to-end driving architecture operates using only camera data, without explicit BEV representation and forgoing the need for LiDAR. PRIX leverages a visual feature extractor coupled with a generative planning head to predict safe trajectories from raw pixel inputs directly. A core component of our architecture is the Context-aware Recalibration Transformer (CaRT), a novel module designed to effectively enhance multi-level visual features for more robust planning. We demonstrate through comprehensive experiments that PRIX achieves state-of-the-art performance on the NavSim and nuScenes benchmarks, matching the capabilities of larger, multimodal diffusion planners while being significantly more efficient in terms of inference speed and model size, making it a practical solution for real-world deployment.

PRIX's architecture processes multi-camera images through a visual backbone featuring our novel CaRT module. These enhanced visual features, combined with the vehicle's state and noisy anchors, are fed into a conditional diffusion planner to generate the final, safe trajectory.

Robustness to Weather Conditions

PRIX demonstrates strong robustness, generating consistent and safe trajectories across clear, rainy, and snowy conditions.

Weather Example

Corresponding Model Predictions (Example 1)

---

Weather Example 2

Corresponding Model Predictions (Example 2)

Trajectory Refinement via Diffusion

The model generates initial trajectories and refines them over diffusion steps. The final selected path is shown in red, with the second-best alternative in blue.

Hyper Parameters

The table below summarizes the full hyperparameter configuration used for the PRIX model. We group settings for the backbone, detection and planning heads, batching and precision, optimization, distributed training, and loss weights.

Additional Experiments

All experiments in this section are performed on Navsim-v1 unless noted otherwise.

---

Backbone Capacity, Speed, and Stability

• Aim: To identify the backbone (ResNet34, ResNet50, ResNet101) that offers the best balance of performance, speed, and stability.
• Method: PRIX was trained with each backbone, and PDMS (mean ± standard deviation over five runs), parameter count, and FPS were measured.
• Result: ResNet34 provides the best overall trade-off. It is the fastest model (57.0 FPS) and the most stable (87.8 ± 0.1 PDMS), while showing only a minor performance difference compared to the larger and slower ResNet101.

---

Ablation on Loss Weights

• Aim: To determine the best weighting for the detection loss and the semantic loss.
• Method: A grid search was performed over different weight combinations, and the PDMS score was recorded for each setting.
• Result: A low detection loss weight and a high semantic loss weight perform best. The configuration with detection weight 1 and semantic weight 10 yields 87.8 PDMS. Performance increases consistently as the semantic loss weight increases.

---

Sensor Failures

• Aim: To test PRIX's robustness to sensor failures, such as camera noise or dropout.
• Method: Sensor failures were simulated at test time for a standard model. New models were then trained with these corruptions (noise or dropout) to evaluate whether robustness improves.
• Result: While failures reduce the baseline model's score from 88.7 to 82.2 PDMS, training with noise recovers the score to 84.7.

---

Impact of Ego Status in the Planning Head

• Aim: To understand how important the ego status input (velocity, acceleration) is for the planner.
• Method: We corrupted the ego status at test time (masking it, replacing it with noise) for both a standard model and a model trained *with* status corruption.
• Result: Ego status is critical, but PRIX can be trained to be robust. Corrupting the status drops the baseline model's score from 87.8 to 66.8 PDMS. However, the model trained with corruption achieves a strong 84.7 PDMS, even with random inputs.