Description
Question 1 [28 Points] (Implementing VQ-VAE) In this question, we implement a simple version of
VQ-VAE architecture proposed in [2]. We train this model on the Fashion MNIST and sample from it by
training a simple autoregressive model for latent distribution learning.
1. Complete Question 1 in Lastname_Firstname_Asgn3.ipynb to implement a vector quantized VAE.
2. Answer all parts marked by # COMPLETE .
Question 2 [32 Points](Diffusion ScoreMatching)In this question, we implement a very basic diffusion
model which invokes diffusion score matching (DSM) along with basic reverse SDE to learn MNIST image
generation. You can learn more about DSM and reverse SDE in [1, 3], though what we had in the lecture
is enough to complete this question.
1. Complete Question 2 in Lastname_Firstname_Asgn3.ipynb .
2. Answer all parts marked by # COMPLETE .
References
[1] Yang Song, Conor Durkan, Iain Murray, and Stefano Ermon. Maximum likelihood training of
score-based diffusion models. In Advances in Neural Information Processing Systems (NeurIPS), 2021.
Available at https://arxiv.org/abs/2101.09258.
[2] Aaron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu. Neural discrete representation learning.
In Advances in Neural Information Processing Systems (NeurIPS), 2017. Available at https://arxiv.org/
abs/1711.00937.
[3] Pascal Vincent. A connection between score matching and denoising autoencoders. Neural Computation, 23(7):1661–1674, 2011.