Description
AMATH 563 Homework 1: Regression, Model Selection and DMD
Download the MNIST data set (both training and test sets and labels): http://yann.lecun.com/exdb/mnist/.
The labels will tell you which digit it is: 1, 2, 3, 4, 5, 6, 7, 8, 9, 0. Let each output be denoted by the vector yj
.
“1” =
1
0
0
.
.
.
0
0
, “2” =
0
1
0
.
.
.
0
0
, · · · , “9” =
0
0
0
.
.
.
1
0
, “0” =
0
0
0
.
.
.
0
1
(1)
Now let B be the set of output vectors
B = [y1 y2 y3 · · · yn] (2)
and let the matrix A be the corresponding reshaped (vectorized) MNIST images
A = [x1 x2 x3 · · · xn] (3)
Thus each vector xj ∈ R
n
2
is an a vector reshaped from the n × n image.
1. Using various AX = B solvers, determine a mapping from the image space to the label space.
2. By promoting sparsity, determine and rank which pixels in the MNIST set are most informative for
correctly labeling the digits. (You’ll have to come up with your own heuristics or empirical rules for
this. Use pcolor to help you visualize the results from X)
3. Apply your most important pixels to the test data set to see how accurate you are with as few pixels as
possible.
4. Redo the analysis with each digit individually to find the most important pixels for each digit.
5. IMPORTANT: Think about the interpretation of what you are doing with this AX = B problem.
This is an exploratory homework. So play around with the data and make sure to make lots of plots. Good
luck, and have fun.
AMATH 563 Homework 2: Dynamics and Model Discovery
Consider the following historical (and classic) data set concerning Canadian lynx and snowshoe hare populations from 1845 to 1903.
Figure 1: Population data.
1. Develop a DMD model to forecast the future population states
2. Do a time-delay DMD model to produce a forecast and compare with regular DMD. Determine if it is
likely that there are latent variables.
3. Empirical Predator-Prey models such as Lotka-Volterra are commonly used to models such phenomenon.
Consider the model x˙ = (b − py)x and y˙ = (rx − d)y. Use the data to fit values of b, p, r and d.
4. Find the best fit nonlinear, dynamical systems model to the data using sparse regression.
5. Compute the KL divergence of the best model fit to the data between all the above models.
6. Retain three of your best fit models and compare their AIC and BIC scores.
Download the data set BZ.mat (which is a snipet from a Belousov-Zhabotinsky chemical oscillator movie –
check them out on youtube).
1. get the data: download from the course website (It is next to HW 2)
2. See what you can do with the data (i.e. repeat the first two steps above)
The following code may be helpful for view the data.
[m,n,k]=size(BZ_tensor); % x vs y vs time data
for j=1:k
A=BZ_tensor(:,:,j);
pcolor(A), shading interp, pause(0.2)
end
This is an exploratory homework. So play around with the data and make sure to make lots of plots. Good
luck, and have fun.
AMATH 563 Homework 3: Extended Yale Faces B Database – Eigenfaces
Yale Faces B
Download two data sets (ORIGINAL IMAGE and CROPPED IMAGES)
Your job is to perform an analysis of these data sets. Please start with the cropped images and
perform the following analysis.
1. Do an SVD analysis of the images (where each image is reshaped into a column vector and
each column is a new image).
2. What is the interpretation of the U, Σ and V matrices?
3. What does the singular value spectrum look like and how many modes are necessary for good
image reconstructions? (i.e. what is the rank r of the face space?)
4. compare the difference between the cropped (and aligned) versus uncropped images.
Face Identification: see if you can build a classifier to identify individuals in the training set.
• (test 1) Face Classification: Consider the various faces and see if you can build a classifier
that can reasonably identify an individual face.
• (test 2) Gender Classification: Can you build an algorithm capable of recognizing men
from women?
• (test 3) Unsupervised algorithms: In an unsupervised way, can you develop algorithms
that automatically find patters in the faces that naturally cluster?
NOTE: You can use any (and hopefully all) of the different clustering and classification methods
discussed. Be sure to compare them against each other in these tasks.
AMATH 563 Homework 4: Neural Networks for Dynamical Systems
Download the accompanying ZIP file which includes MATLAB code for solving (i) A reaction-diffusion system of equations, and (ii) The Kuramoto-Sivashinsky (KS) equation.
1. Train a NN that can advance the solution from t to t + ∆t for the KS equation
2. Compare your evolution trajectories for your NN against using the ODE time-stepper provided with
different initial conditions
3. For the reaction-diffusion system, first project to a low-dimensional subspace via the SVD and see how
forecasting works in the low-rank variables.
For the Lorenz equations (code given out previously in class emails), consider the following.
1. Train a NN to advance the solution from t to t + ∆t for ρ = 10, 28 and 40. Now see how well your NN
works for future state prediction for ρ = 17 and ρ = 35.
2. See if you can train your NN to identify (for ρ = 28) when a transition from one lobe to another is
imminent. Determine how far in advance you can make this prediction. (NOTE: you will have to label
the transitions in a test set in order to do this task)
