UniINR is an advanced neural framework designed to handle rolling shutter correction, deblurring, and interpolation in a unified approach guided by event-based inputs. This repository provides the code, datasets, and pre-trained models for reproducing results from our ECCV 2024 paper.

Our framework, UniINR, is aimed at enhancing video quality by addressing key challenges that arise in event-guided imaging, such as rolling shutter distortion, motion blur, and interpolation between frames. This repository includes all essential tools to train, test, and extend our approach.

• Video Demo: YouTube Link
• Paper: Arxiv PDF

For quicker experiments, we provide pre-trained models along with their visualization results and log files. Access them through the provided download links, which should assist in replication and further experiments. Release Link: OneDrive

Our work is built on two primary datasets used for training and testing:

1. DeepUnrollNet Dataset (GitHub Link)
   • Since the original dataset might no longer be accessible, I provide this backup link.
2. EvUnroll Dataset (GitHub Link)

Additionally, our training incorporates:

• GEV Dataset: Available for download on Baidu
• Fastec Dataset: Fastec is segmented for training and testing:
  • Fastec Training Data
  • Fastec Test Data

For generating new rolling shutter and blurred datasets, use the following scripts. Please note that both event simulation and rolling shutter simulation take the original high-frame-rate video as input.

1. Rolling Shutter Blurred Frames:

   tools/1-rs-blur-dataset-generation/generate_rs_blur_frames_fastec.sh

2. Rolling Shutter Sharp Frames:

   tools/1-rs-blur-dataset-generation/generate_rs_sharp_frames_fastec.sh

Or, customize dataset generation with:

python tools/1-rs-blur-dataset-generation/generate_rs_blur_frames.py \
    --dataset_path="./dataset/2-Fastec-Simulated/Train/" \
    --blur_accumulate_frames=<level of blur> \
    --blur_accumulate_step=<rolling step size> \
    --dataset="Fastec"

To begin training, use the following command structure:

python egrsdb/main.py \
  --yaml_file=<YAML_FILE> \
  --log_dir=<LOG_DIR> \
  --alsologtostderr=True

Ensure that the specified YAML_FILE contains the correct configuration for your experiment setup.

If you find UniINR helpful in your research, please consider citing our paper:

@inproceedings{yunfanuniinr,
    title = {UniINR: Event-guided Unified Rolling Shutter Correction, Deblurring, and Interpolation},
    author = {Yunfan, LU and Liang, Guoqiang and Wang, Yusheng and Wang, Lin and Xiong, Hui},
    booktitle = {European Conference on Computer Vision (ECCV)},
    year = {2024},
}

We would like to thank the authors of the following works for making their datasets available, which greatly facilitated our research:

• DeepUnrollNet (GitHub Link)
• EvUnroll (GitHub Link)