Leveraging Foundation Vision Models with QLoRA and DoRA for Dairy Cattle Behavior Recognition

This repository contains the official implementation of our paper exploring Parameter-Efficient Fine-Tuning (PEFT) methods for adapting billion-parameter vision models to livestock behavior classification.

• 83.16% test accuracy using QLoRA with only 2.72% trainable parameters
• 65% reduction in training time compared to training from scratch
• 1:98 training-to-test ratio - 2,160 verified images generalize to 211,800+ real-world samples
• Underfitting, not overfitting, is the primary challenge when adapting foundation models to agricultural imagery

PEFT-Fine-tuning-cows/
├── assets/
│   └── infographics.png
├── notebooks/
│   ├── DoRA Fewer Layers R=8.ipynb
│   ├── DoRA Fewer Layers R=16.ipynb
│   ├── DoRA More Layers R=16.ipynb
│   ├── DoRA More Layers R=64.ipynb
│   ├── Q-lora Fewer Layers R = 8.ipynb
│   ├── Q-lora Fewer Layers R = 16.ipynb
│   ├── Q-lora More Layers Rank = 16.ipynb
│   ├── Q-lora More Layers Rank = 64.ipynb
│   ├── TrainFromScatch_Preprocesing Pipeline Val = 1.ipynb
│   └── UsingPretrainedModel_DinoV3 Embeddings Val = 1.ipynb
├── environment.yml
├── requirements.txt
├── LICENSE
└── README.md

Model checkpoints are available on Hugging Face:

from transformers import AutoModel
from peft import PeftModel

# Load QLoRA model (best performing)
base_model = AutoModel.from_pretrained("facebook/dinov3-vit7b16-pretrain-lvd1689m")
model = PeftModel.from_pretrained(base_model, "Sonam5/cow-behavior-dinov3-qlora-r64")

Our dataset consists of nine dairy cow behaviors:

Total: 2,160 training images → 211,800 test samples (1:98 ratio)

• Python 3.12
• CUDA compatible GPU with 16GB+ VRAM (Tesla V100 or equivalent)

# Clone the repository
git clone https://github.com/YOUR_USERNAME/PEFT-Fine-tuning-cows.git
cd PEFT-Fine-tuning-cows

# Create conda environment
conda env create -f environment.yml
conda activate cow-behavior-analysis

# Clone the repository
git clone https://github.com/YOUR_USERNAME/PEFT-Fine-tuning-cows.git
cd PEFT-Fine-tuning-cows

# Install dependencies
pip install -r requirements.txt

If you find this work useful, please cite our paper:

@article{yang2025peft_cattle,
  title={When Billion-Parameter Foundation Models Meet Limited Data: Parameter-Efficient Fine-Tuning with QLoRA and DoRA for Generalizable Image Classification},
  author={Yang, Haiyu and Sharma, Sumit and Liu, Enhong and Hostens, Miel},
}

Contributions are welcome! Please feel free to submit a Pull Request.

This project is licensed under the MIT License - see the LICENSE file for details.

• DINOv3 - Foundation model
• PEFT - Parameter-efficient fine-tuning library
• bitsandbytes - Quantization library

**Haiyu Yang ** - Cornell University