Description

This Pull Request builds on top of #3104 and #3098, adding a new autocast sub-config to the inference: block of the PyTorch configuration.

The use of torch.autocast during inference was first introduced in #3012 (“Shaokai/faster inference”)
with the goal of improving inference speed.
However, we later discovered that enabling torch.autocast can significantly decrease inference accuracy — this is the root cause of issue #3083 and the forum discussion “Better results with DeepLabCut==2.3.0 than DeepLabCut==3.0 with PyTorch”.

Key Changes

• Introduced an autocast sub-config in the inference: block of pytorch_config.yaml.
• Added an AutocastConfig class to manage parameters consistently with the existing compile and multithreading options.
• Inference logic now respects this configuration and defaults to autocast disabled to avoid unintended accuracy drops.
• Advanced users can enable autocast either:
  • programmatically (by passing an InferenceConfig object or a dictionary), or
  • by editing the pytorch_config.yaml file under the inference: block.

Notes & Future Work

• Default behavior is changed back to v3.0.0rc8 — autocast is now off by default to preserve accuracy.
• Advanced users may still enable it to experiment with potential speedups, accepting the risk of reduced prediction quality.
• In future work, we plan to investigate whether training models with torch.autocast can mitigate this accuracy gap and allow safer use of autocast at inference.