Description
Question 1 [30 Points] (Projectile Example) We implement the linear model that estimates the landing
point of the projectile in our dummy example. To this end, we need to make a dataset, write a model
and train it.
1. Start your code by writing a class called LinearMachine() . This is given in the attached notebook.
You may complete all functions of this class in the notebook.
We know the model and training loop from the written question. We just need to learn how to generate
a dataset. Read the following text to learn how to do it.
Generating Synthetic Dataset As we are not really going to do the experiment with a projectile and
measure distances, we generate synthetic data. This means that we generate random data-points and
assign them labels that are theoretically correct. This is in contrast to real-world data that would have been
collected by really doing the projectile experiment and measure distances. To generate our synthetic
dataset, we use random module in NumPy to generate some random heights and velocities. We then
determine the label for each height and velocity using the analytical expression d = 0.45v
√
d.
2. Write a function that takes dataset size I, mean velocity v0, velocity variance σ
2
v
, mean height h0,
height variance σ
2
h
as input and returns a synthetic dataset.
• Heights and velocities are generated i.i.d. as vi = |v˜i
| and hi = |h˜
i
|
→ v˜i
is a normal random variable with mean v0 and variance σ
2
v
→ h˜
i
is normal random variables with mean h0, and variance σ
2
h
Call this function data_synthesizer .
3. Write a function that takes the dataset and learning rate η, computes the empirical risk, and trains
the linear model by minimizing the empirical risk using the gradient descent algorithm. Call this
function train_GD.
Hint: Note that you have already computed the gradient in Part 2 of Written Question 1.
4. Write a function that takes a dataset and returns the minimizer of the empirical risk. Call this
function train .
Hint: Note that you have already computed the gradient in Part 3 of Written Question 1.
5. Use the implemented class to generate a synthetic dataset with
→ I = 100, v0 = 1, σ
2
v = 5, h0 = 3, and σ
2
h = 3.
Write a function (you may implement it outside the class) that gets an array of learning rates
and returns the Euclidean distance (norm of the difference) between the outputs of train and
train_GD for each learning rate in the array. Plot the difference against the vector of learning rates.
Now that we have trained the model, we should test it. Read the following text to learn how to do it.
Testing Trained Model For testing, we must generate a separate synthetic dataset of size J and inspect
the performance of our model by comparing its outputs with the labels of the dataset. This means that
we should determine the empirical risk with our new test dataset and evaluate its value for the trained
model parameters.
6. Write a function that takes a size for the test dataset and returns the empirical risk of the trained
model for the test dataset. Call this function test .
Hint: Note that based on description test and training datasets are independent.
7. Write a function (you may implement it outside the class) that gets the values
→ T, I J, as well as v0, σ
2
v
, h0, and σ
2
h
,
and returns the average test risk by looping over T independent realizations of training and test
datasets.
8. Set the function inputs to
→ T = 100, J = 10, v0 = 1, σ
2
v = 5, h0 = 3, and σ
2
h = 3.
Evaluate the average test risk for an array of training dataset sizes, i.e., an array of I, and plot the
test risk against I.
Question 2 [30 Points] (Pattern Recognition) In this exercise, we train a single perceptron to recognize
a pattern in an image. You may take a look at the perceptron algorithm in Chapter 1. We first make a
minor modification to the algorithm. For this, read the following text.
Introducing Learning Rate We can introduce a learning rate to the algorithm by modifying its update
rule as w ← w − η sign (zi) xi
, where η is the learning rate.
With this modification in mind, answer the following items:
1. Define a class called PerceptronMachine() for a perceptron with 9 inputs. You may use the
code given in the notebook.
→ As an attribute of this class, we should define the weights and bias. Yo may initiate them with
some random numbers.
2. Write a function that passes data-point x ∈ R9
through the perceptron. Call this function forward .
3. Write a function that gets a dataset and trains the perceptron using the perceptron algorithm. Call
this function train .
We next need to generate a dataset. Read the following text to learn how to do it.
Image as List As mentioned in the lecture, we can represent any image by an array of pixels. For this
exercise, we consider a very simple example:
a 3 × 3 image consists of 9 pixels. We consider a simple case, where each pixel has either value 0 (black)
or value 256 (white). We can visualize this image using Matplotlib .
A sample code is provided in the notebook to plot the following 9-pixel pattern of X.
We can show this image in a list of size 9, where each entry of this list is the value of the pixel. The index
of each entry has been specified in the above image. For instance the above X pattern is specified by the
list x = 256 * [0, 1, 0, 1, 0, 1, 0, 1, 0] .
4. Write a function that generates the following dataset: the set of all 9-pixel images (each pixel either
black or white), where the label of X pattern is 1 and the label of others is 0. You may note the
following items
→ We have 2
9 = 512 binary vectors of dimension 9; thus, we have 512 images in the dataset.
→ Label x = 256 * [0, 1, 0, 1, 0, 1, 0, 1, 0] with y=1 and other lists with y=0 .
5. Train the implemented perceptron with this new dataset.
6. Validate your training by printing the output of the perceptron to X pattern and 10 other randomly
selected images from the dataset.
Hint: This problem is linearly separable; thus, the perceptron should exactly learn this classifier.