Official implementation of "Towards Explainable Video Camouflaged Object Detection: SAM2 with Eventstream-Inspired Data" (AAAI 2026).

EventVCOD introduces a novel framework for Video Camouflaged Object Detection (VCOD) by leveraging event camera-inspired data and Segment Anything Model 2 (SAM2). Our approach provides explainable detection through event-based motion representations that capture temporal dynamics invisible to standard RGB cameras.

• 🎯 SAM2-based Architecture: Fine-tuned SAM2 with custom prompt generators for VCOD
• ⚡ Event-Inspired Data: Novel eventstream-like representation from RGB videos
• 🎬 Video Understanding: Temporal coherence through event polarity (+/-) encoding
• 🔍 Explainable Detection: Motion-based interpretable features
• 📊 State-of-the-art Performance: Superior results on MoCA-Mask-Video and CAD-2016 benchmarks

Our framework consists of three main components:

1. Event-Inspired Data Generation: Convert RGB frames to event-like representations with positive/negative polarities
2. Prompt Generator: Dense embedding generator for SAM2 conditioning
3. SAM2 Backbone: Fine-tuned image encoder with memory attention mechanism

• Python 3.8+
• CUDA 12.1+
• PyTorch 2.3.0+

# Clone the repository
git clone https://github.com/yourusername/EventVCOD.git
cd EventVCOD

# Install dependencies
pip install -r requirements.txt

# Install SAM2
pip install -e .

Main dependencies include:

• torch>=2.3.0
• torchvision>=0.18.0
• opencv-python>=4.8.0
• numpy>=1.24.2
• Pillow>=9.4.0
• tensorboardX>=2.6.2
• timm==0.4.12

See requirements.txt for complete list.

All datasets and pre-processed event-like data are available at:

• 🔗 Netdisk

Organize your datasets as follows:

datasets/
├── MoCA-Video-Train_event/
│   ├── crab/
│   ├── flatfish_0/
│   └── ...
├── MoCA-Video-Test_event/
│   ├── arctic_fox/
│   ├── black_cat_1/
│   └── ...
└── CAD2016_event/
    └── ...

To generate event-like representations from your own RGB videos:

cd data_manipulate
python eventflow_like_gen_claude.py --input_dir /path/to/videos --output_dir /path/to/output

All pre-trained models (including SAM2 checkpoints and fine-tuned EventVCOD models) are available at:

• 🔗 Netdisk

Download the checkpoints and place them in the checkpoints/ directory.

python train.py --config sam2/configs/sam2.1_training/sam2.1_hiera_b+_VCOD_finetune_tiny_adp0317_video_part2_SAM2_finetune30.yaml

# 4 GPUs example
python -m torch.distributed.launch \
    --nproc_per_node=4 \
    --master_port=29500 \
    train.py \
    --config sam2/configs/sam2.1_training/sam2.1_hiera_b+_VCOD_finetune_tiny_adp0317_video_part2_SAM2_finetune30.yaml \
    --name eventvcod_exp \
    --tag experiment_v1

Main training parameters in config files:

• resolution: Input resolution (default: 1024)
• train_batch_size: Batch size per GPU
• num_frames: Number of frames per video clip
• num_epochs: Total training epochs
• base_lr: Base learning rate

Modify configurations in sam2/configs/sam2.1_training/ for different settings.

python test.py \
    --config sam2/configs/sam2.1/sam2.1_hiera_b+_VCOD_infer_modify.yaml \
    --model /path/to/checkpoint.pth

We evaluate using standard VCOD metrics:

• S-measure (Sα): Structure similarity
• E-measure (Eφ): Enhanced alignment measure
• weighted F-measure (Fwβ): Weighted precision-recall
• MAE (M): Mean Absolute Error
• mean Dice (mDice): Dice coefficient
• mean IoU (mIoU): Intersection over Union

For comprehensive evaluation using MATLAB:

# For MoCA-Mask-Video dataset
cd eval
run main_MoCA.m

# For CAD-2016 dataset
run main_CAD.m

cd eval/PySODMetrics
python evaluate.py --pred_dir /path/to/predictions --gt_dir /path/to/ground_truth

Note: ↑/↓ indicates metrics not reported in original papers. Full quantitative results available in the paper.

EventVCOD/
├── sam2/                           # SAM2 core implementation
│   ├── modeling/                   # Model architectures
│   ├── configs/                    # Configuration files
│   │   ├── sam2.1_training/       # Training configs
│   │   └── ablation/              # Ablation study configs
│   └── utils/                      # Utility functions
├── training/                       # Training pipeline
│   ├── trainer.py                 # Main trainer
│   ├── trainer_supervision.py     # Supervised training
│   └── model/                     # Model definitions
├── datasets/                       # Dataset implementations
│   ├── datasets.py                # Dataset loaders
│   └── transform_custom.py        # Data augmentation
├── data_manipulate/               # Event data generation
│   ├── eventflow_like_gen_claude.py  # Event generation
│   └── eventflow_p_n_visualization.ipynb  # Visualization
├── eval/                          # Evaluation scripts
│   ├── PySODMetrics/             # Python metrics
│   └── *.m                       # MATLAB evaluation
├── train.py                       # Training entry point
├── test.py                        # Testing entry point
└── utils.py                       # General utilities

Our event data generation process:

1. Frame Difference: Compute temporal derivatives between consecutive frames
2. Polarity Assignment: Threshold-based positive/negative event classification
3. Accumulation: Aggregate events over time windows
4. Normalization: Scale to appropriate intensity ranges

Example usage:

from data_manipulate.eventflow_like_gen_claude import generate_events

events_pos, events_neg = generate_events(
    video_path='path/to/video.mp4',
    threshold=0.2,
    output_dir='path/to/output'
)

See data_manipulate/eventflow_p_n_visualization.ipynb for visualization examples.

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

This work builds upon:

• SAM2 - Meta's Segment Anything Model 2

• 📄 Paper PDF
• 📑 Supplementary Material
• 💾 Datasets & Checkpoints (Netdisk)

Star ⭐ this repository if you find it helpful!