COM507 – Optional Research Project in Communication Systems

Efe Tarhan, MSc Student in Communication Systems

Supervisor: Dongqing Wang, Image and Visual Representation Lab (IVRL)

This repository implements a NeRF editing framework that enables localized relighting and texture edits using pretrained diffusion models. Built on the DDS pipeline, it integrates wavelet-based gradient filtering to preserve reflections and fine appearance details during editing. The framework keeps scene geometry fixed after NeRF training, ensuring structural consistency, and enhances edits with surface normal prediction for improved view-dependent rendering. The result is high-fidelity, semantically guided 3D scene edits with strong reflection and color consistency.

.
├── README.md
├── requirements.txt
├── assets/                          # Diagrams and logos
├── nerfstudio/3d_editing/dc_nerf/  # NeRF editing module
│   ├── data/                       # Custom datamanagers & parsers
│   ├── engine/                     # Trainer
│   ├── fields/                     # Fields used in models
│   ├── models/                     # Nerfacto/Splatfacto variants
│   └── pipelines/                  # Base + DC pipelines
├── dc/                             # Diffusion controller 
│   ├── dc.py, 
|   ├── cds.py                      # Main training logic
│   ├── dc_unet.py                  # FreeU UNet
│   └── utils/                      # Wavelet, image, FreeU utils

First clone the repository

git clone https://github.com/tarhanefe/DDSNeRFRelight.git
cd DDSNeRFRelight

Create a conda environment

conda create -n relight python=3.9
conda activate relight

Then, install the required packages.

pip install torch==2.1.2+cu118 torchvision==0.16.2+cu118 --extra-index-url https://download.pytorch.org/whl/cu118
pip install numpy==1.26.4
conda install -c "nvidia/label/cuda-11.8.0" cuda-toolkit
pip install ninja git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch
pip install -e .

For 3D editing, you need to install the following packages.

cd 3d_editing
pip install -e .
pip install gsplat==0.1.6
pip install huggingface_hub==0.21.0
pip install tyro==0.6.6
pip install torchwavelets

In case the following command won't work. Manually register the dc method to the datamanager config in the main environment nerfstudio library.

ns-train -h

You can access the dataset used for training and evaluation via the following link:
🔗 Google Drive Dataset

Put the data folders under the ./3d_editing folder.

First train the scene using the default NeRF constructor of the nerfstudio

ns-train nerfacto --data ./gardenspheres_n nerfstudio-data --downscale-factor 8

Following that use the given command to edit the scene.

ns-train dc --data ./gardenspheres_n --load-dir ./outputs/gardenspheres_n/nerfacto/2025-03-30_013255/nerfstudio_models/ \
 --pipeline.dc.src_prompt "a photo of two reflective spheres" \
 --pipeline.dc.tgt_prompt "a photo of two reflective green spheres" \
 --vis viewer \
 --max_num_iterations 200 \
 --pipeline.dc.pipeline dc \
 --pipeline.dc.wavelet_filtering True \
 --pipeline.dc.wavelet_name db4 \
 --pipeline.dc.wavelet_level 2 \
 --pipeline.dc.guidance-scale 7.5 nerfstudio-data --downscale-factor 8

The following parameters in DCConfig control the behavior of the diffusion-guided editing pipeline:

These parameters are set inside the editing pipeline and can be overridden via CLI using --pipeline.dc.<parameter> flags.