Attaching controlNet into Stable Diffusion to finetune the model ControlNet Architecture 1. Using Circle Fill50K Dataset fusing/fill50k SDXL Results Before Finetuning ControlNet Results After Finetuning via condition 1 After Finetuning via condition 2 2. Using Pose Dataset merlinyx/pose-controlnet SDXL Results Before Finetuning ControlNet Results After Finetuning via condition 1 After Finetuning via condition 2 Disclaimer: All of the code is inside the Jupyter notebook file. Parts of the code are copied from unsolved assignments of Prof. Minhyuk Sung @ KAIST, a graduate course about diffusion models. Work is done during KAUST internship with KAUST GPUs @ Prof. Mohamed Elhoseiny Lab, as part of the learnings to create a diffusion project.
