This repository contains the code, models, and supporting materials for a research project using a modified Pix2Pix model to synthesize depth-aware focal stacks from a single image input. The system uses a binocular (dual-camera) setup: with an EDoF (Extended Depth of Field) lens and a varifocal camera to generate depth-variant imagery. This project builds on the original pytorch-CycleGAN-and-pix2pix model, extending it for generating varifocal image stacks from a single EDoF input.
Presented and discussed in greater detail in the research paper:
ADD PAPER AND SUPPLEMENT HERE
Data is presented as: EDoF image | Generated images | Ground truth images
-
512x512 Model Results
-
1024x1024 Model Results
-
No Slice Model Results
- Create and activate virtual environment:
conda create --name pix2pix python=3.9
conda activate pix2pix- Install dependencies via:
- requirements.txt
Or:
conda install torch
conda install torchvision
pip install dominate
pip install visdom
pip install wandb- Clone this repo:
git clone https://github.com/sccanright/varifocal_pix2pix
cd varifocal_pix2pix- Pre-trained models are available for download and unzipping
- release: models
- Make sure it is saved within the varifocal_pix2pix folder
- release: models
# Download the checkpoints model release:
wget https://github.com/sccanright/varifocal_pix2pix/releases/download/models/checkpoints.zip -O checkpoints.zip
# Unzip the checkpoints.zip file
unzip checkpoints.zip
# Remove the checkpoints.zip file after extraction
rm checkpoints.zip-
Test datasets are available for download and unzipping
- release: data
- Make sure they are saved within the varifocal_pix2pix/datasets folder
- release: data
-
Download and unzip the test sets to ./datasets:
# Download zipped folders
wget https://github.com/sccanright/varifocal_pix2pix/releases/download/data/512_SLICED_FIELDSET.zip -O datasets/512_SLICED_FIELDSET.zip
wget https://github.com/sccanright/varifocal_pix2pix/releases/download/data/1024_SLICED_FIELDSET.zip -O datasets/1024_SLICED_FIELDSET.zip
wget https://github.com/sccanright/varifocal_pix2pix/releases/download/data/NO_SLICE_FIELDSET.zip -O datasets/NO_SLICE_FIELDSET.zip
# Unzip to the correct folder
unzip datasets/512_SLICED_FIELDSET.zip -d datasets/
unzip datasets/1024_SLICED_FIELDSET.zip -d datasets/
unzip datasets/NO_SLICE_FIELDSET.zip -d datasets/
# Remove the .zip files after extraction
rm datasets/512_SLICED_FIELDSET.zip
rm datasets/1024_SLICED_FIELDSET.zip
rm datasets/NO_SLICE_FIELDSET.zip- Run the tests:
- !! Only run one model at a time !!
# Testing trained models
python test.py --dataroot ./datasets/512_SLICED_FIELDSET --name 512slicedata01 --model pix2pix --gpu_ids 0 --netG unet_512 --input_nc 3 --output_nc 30
python test.py --dataroot ./datasets/1024_SLICED_FIELDSET --name 1024slicedata01 --model pix2pix --gpu_ids 0 --netG unet_1024 --input_nc 3 --output_nc 30
python test.py --dataroot ./datasets/NO_SLICE_FIELDSET --name noslicedata01 --model pix2pix --gpu_ids 0 --netG unet_1024 --input_nc 3 --output_nc 30- Locate results:
- ./results/NAME/test_latest
- Train a new model: (Update netG and input/output channels as needed)
python train.py --dataroot ./datasets/FIELDSET --name NAMEofMODEL --model pix2pix --gpu_ids 0 --netG unet_1024 --input_nc 3 --output_nc 30- Linux, macOS, or Windows with WSL
- Python 3.8+
- CPU or NVIDIA GPU + CUDA CuDNN
If you use this code in your research, please cite both this repository and the original Pix2Pix project:
@misc{canright2025varifocalpix2pix,
author = {Slater Canright},
title = {varifocal Camera Pix2Pix Model},
year = {2025},
url = {https://github.com/sccanright/varifocal_pix2pix}
}
@inproceedings{isola2017image,
title={Image-to-image translation with conditional adversarial networks},
author={Isola, Phillip and Zhu, Jun-Yan and Zhou, Tinghui and Efros, Alexei A},
booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition},
pages={1125--1134},
year={2017}
}- ButtonCapture.py
- crop.py
- slice.py
- reorganize_data.py
- analyze.py
- exposure_test.py
- GPIOTest.py
- GPIOZeroTest.py
- Camera Box
- Box Lid
- EDoF Holder
- EDoF Lid
- Lens Mount - Left
- Lens Mount - Right
- SolidWorks and extra code
# Clone just the extras branch directly:
git clone --branch extras --single-branch https://github.com/sccanright/varifocal_pix2pixOur code comes directly from pytorch-CycleGAN-and-pix2pix, with slight script modifications to function with our expanded data requirements.