Authors: Yushi Guan, Daniel Kwan, Jean Sebastien Dandurand, Xi Yan, Ruofan Liang, Yuxuan Zhang, Nilesh Jain, Nilesh Ahuja, Selvakumar Panneer, Nandita Vijaykumar
ICLR 2025 Spotlight

🌐 Project Page | 📝 OpenReview

Retri3D is a solution for the retrieval of 3D neural graphics representations, supporting retrieval of both pre-trained NeRF and 3DGS models. Retri3D does not require the access to the original training data, and works with pre-trained models by distinguishing the clean content from the artifacts.

This project builds upon the nerfstudio codebase. Follow these steps to set up the environment:

1. Create and activate a conda environment:

conda create --name retri3d python=3.10
conda activate retri3d

2. Install PyTorch and other dependencies:

conda install -c "nvidia/label/cuda-11.8.0" cuda-toolkit
pip install torch==2.1.2+cu118 torchvision==0.16.2+cu118 --extra-index-url https://download.pytorch.org/whl/cu118
pip install ninja git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch
pip install -r requirements.txt

3. Install nerfstudio:

cd nerfstudio
pip install --upgrade pip setuptools
pip install -e .

4. Install lerf

cd lerf
python -m pip install -e .

5. Install LLaVA

cd LLaVA
pip install -e .

6. Install xdecoder

cd xdecoder
pip install -e .
pip install -r assets/requirements/requirements_no_versions.txt

8. Install other dependencies

conda install -c conda-forge mpi4py openmpi
sudo apt install colmap
sudo apt install imagemagick
sudo apt install ffmpeg

1. We have already prepared the lerf dataset under:

data_samples/lerf_dataset/

To run the retrieval experiment on lerf dataset with our SMCC method, run:

python runner.py --scene_num=13

You can adjust the arguments in runner.py for alternative settings. By default, the results are stored in the exp_dir directory.