Description
1 Generative Adversarial Networks (50 points)
We will be training Generative Adversarial Networks (GAN) on the CUB 2011 Dataset.
• Setup: Run the following command to setup everything you need for the assignment:
./setup.sh /path/to/python env/lib/python3.8/site-packages
• Question: Follow the instructions in the README.md file in the gan/ folder to complete the implementation of GANs.
• Deliverables: The code will log plots to gan/data gan, gan/data ls gan, and gan/data wgan gp.
Extract plots and paste them into the appropriate section below. Note for all questions, we ask for final
FID. Final FID is computed using 50K samples, at the very end of training. See the final print out for
”Final FID (Full 50K): ”.
• Debugging Tips:
– GAN losses are pretty much meaningless! If you want to understand if your network is learning,
visualize the samples. The FID score should generally be going down as well.
– Do NOT change the hyper-parameters at all, they have been carefully tuned to ensure the networks will train stably. If things aren’t working its a bug in your code.
– For debugging, disable JIT using export PYTORCH JIT=0 python … and disable
AMP by using the flag –disable amp. However, do note that disabling JIT will cause
the FID calculation to fail. So only disable JIT to make sure that your network code runs correctly, then re-enable when training. If you observe any errors involving type mismatches and
tensors that have half types, it is due to AMP, you may need to explicitly cast the tensor using
.half().
– Here is a sample image from WGAN-GP at the end of training. The other networks may have
variations but should look similar:
• Expected results:
– Vanilla GAN: Final FID should be less than 110.
– LS-GAN: Final FID should be less than 90.
– WGAN-GP: Final FID should be less than 70.
2 of 7
Homework 2: Generative Modelling 16824
1. Paste your plot of the samples and latent space interpolations from Vanilla GAN as well as the final FID
score you obtained.
FID: TODO: fill in the FID here.
Image
(a) Samples
Image
(b) Latent Space Interpolations
2. Paste your plot of the samples and latent space interpolations from LS-GAN as well as the final FID
score you obtained.
FID: TODO: fill in the FID here.
Image
(a) Samples
Image
(b) Latent Space Interpolations
3. Paste your plot of the samples and latent space interpolations from WGAN-GP as well as the final FID
score you obtained.
FID: TODO: fill in the FID here.
Image (a) Samples Image (b) Latent Space Interpolations
3 of 7
Homework 2: Generative Modelling 16824
2 Variational Autoencoders (30 pts)
We will be training AutoEncoders and Variational Auto-Encoders (VAE) on the CIFAR10 dataset.
• Question: Follow the instructions in the README.md file in the vae/ folder to complete the implementation of VAEs.
• Deliverables: The code will log plots to different folders in vae. Please paste the plots into the appropriate place for the questions below. Note for ALL questions, use the reconstructions and samples
from the final epoch (epoch 19).
• Debugging Tips:
– Make sure the auto-encoder can produce good quality reconstructions before moving on to the
VAE. While the VAE reconstructions might not be clear and the VAE samples even less so, the
auto-encoder reconstructions should be very clear.
– If you are struggling to get the VAE portion working: debug the KL loss independently of the
reconstruction loss to ensure the learned distribution matches standard normal.
• Expected results:
– AE: reconstruction loss should be < 40, reconstructions should look similar to original image.
– VAE: reconstruction loss should be < 145 (β = 1 case).
– VAE: reconstruction loss should be < 125 when annealing β.
4 of 7
Homework 2: Generative Modelling 16824
1. Autoencoder: For each latent size, paste your plot of the reconstruction loss curve and reconstructions.
Image
(a) Loss: latent size 16
Image
(b) Loss: latent size 128
Image
(c) Loss: latent size 1024
Image
(d) Reconstructions: latent size 16
Image
(e) Reconstructions: latent size 128
Image
(f) Reconstructions: latent size 1024
2. VAE: Choose the β that results in the best sample quality, β
∗
. Paste the reconstruction and kl loss curve
plots as well as the sample images corresponding to the VAE trained using constant β
∗
and the VAE
trained using β annealing scheme with β
∗
.
Image
(a) Recon. Loss: constant β
∗
Image
(b) KL Loss: constant β
∗
Image
(c) Samples: constant β
∗
Image
(d) Recon. Loss: β annealed
Image
(e) KL Loss: β annealed
Image
(f) Samples: β annealed
5 of 7
Homework 2: Generative Modelling 16824
3 Diffusion Models (20 points)
We will be running inference using a pre-trained diffusion model (DDPM) on CIFAR-10.
• Setup: Download our pre-trained checkpoint for DDPM from https://drive.google.com/file/d/1gtn9Jv9jBUol7iJw94hw4j6KfpG3SZE/view?usp=sharing.
• Question: Follow the instructions in the README.md file in the diffusion/ folder to complete
the implementation of the sampling procedures for Diffusion Models.
• Deliverables: The code will log plots to diffusion/data ddpm and diffusion/data ddim.
Extract plots and paste them into the appropriate section below.
• Expected results:
– FID of less than 60 for DDPM and DDIM
1. Paste your plots of the DDPM and DDIM samples.
Image
(a) DDPM Samples
Image
(b) DDIM Samples
2. Paste in the FID score you obtained from running inference using DDPM and DDIM.
DDPM FID: TODO: fill in the FID here.
DDIM FID: TODO: fill in the FID here.
6 of 7
Homework 2: Generative Modelling 16824
Collaboration Survey Please answer the following:
1. Did you receive any help whatsoever from anyone in solving this assignment?
⃝ Yes
⃝ No
• If you answered ‘Yes’, give full details:
• (e.g. “Jane Doe explained to me what is asked in Question 3.4”)
2. Did you give any help whatsoever to anyone in solving this assignment?
⃝ Yes
⃝ No
• If you answered ‘Yes’, give full details:
• (e.g. “I pointed Joe Smith to section 2.3 since he didn’t know how to proceed with Question 2”)
3. Note that copying code or writeup even from a collaborator or anywhere on the internet violates the
Academic Integrity Code of Conduct.
7 of 7