Description
Dataset selection:
In your project, you will pick a dataset (not the same as in the previous homeworks) and
describe the problem you would like to solve (classification or regression). Include a link to the
dataset source. It is highly recommended that you pick a dataset with at least 10,000 (or more
observations). There are many ways of describing a big dataset and one way to describe it is – a
big dataset is more complex. Complexity can refer to the number of observations, features, or
the type of data. For this project, there is no restriction on the number of features your dataset
has. However, having more features gives you greater ability to apply the techniques discussed
in class.
Part 1 (50 points)
In this part you will implement a neural network from scratch. You cannot use any existing
Deep Learning Framework. You can utilize NumPy and Pandas libraries to perform efficient
calculations. Refer to Lecture 5 slides for details on computations required.
Write a Class called NeuralNetwork that has at least the following methods (you are free to add
your own methods too):
a. Initialization method.
b. Forward propagation method that performs forward propagation calculations.
c. Backward propagation method that implements the backpropagation algorithm
discussed in class.
d. Train method that includes the code for gradient descent.
e. Cost method that calculates the loss function.
f. Predict method that calculates the predictions for the test set.
Test your NeuralNetwork Class with the dataset you selected. If the dataset is big, you may
notice inefficiencies in runtime. Try incorporating different versions of gradient descent to
improve that (Minibatch, Stochastic etc.). You may choose to use only a subset of your data for
this task (or any other technique). Explain which technique you followed and why.
Part 2 (50 points)
In this part you will implement a 2-layer neural network using any Deep Learning Framework
(e.g., TensorFlow, PyTorch etc.).
You should pick a Deep Learning Framework that you would like to use to implement your 2-
layer Neural Network.
Task 1 (5 points): Assuming you are not familiar with the framework, in this part of the
homework you will present your research describing the resources you used to learn the
framework (must include links to all resources). Clearly explain why you needed a particular
resource for implementing a 2-layer Neural Network (NN). (Consider how you will keep track of
all the computations in a NN i.e., what libraries/tools do you need within this framework.)
For example, some of the known resources for TensorFlow and PyTorch are:
https://www.tensorflow.org/guide/autodiff
https://www.tensorflow.org/api_docs/python/tf/GradientTape
https://pytorch.org/tutorials/beginner/basics/buildmodel_tutorial.html
Hint: You need to figure out the APIs/packages used to implement forward propagation and
backward propagation.
Task 2 (35 points): Once you have figured out the resources you need for the project, you
should design and implement your project. The project must include the following steps (it’s
not limited to these steps):
1. Exploratory Data Analysis (Can include data cleaning, visualization etc.)
2. Perform a train-dev-test split.
3. Implement forward propagation (clearly describe the activation functions and other
hyper-parameters you are using).
4. Compute the final cost function.
5. Implement gradient descent (any variant of gradient descent depending upon your
data and project can be used) to train your model. In this step it is up to you as someone
in charge of their project to improvise using optimization algorithms (Adams, RMSProp
etc.) and/or regularization. Experiment with normalized inputs i.e. comment on how
your model performs when the inputs are normalized.
6. Present the results using the test set.
NOTE: In this step, once you have implemented your 2-layer network you may increase and/or
decrease the number of layers as part of the hyperparameter tuning process.
Task 3 (10 points): In task 2 describe how you selected the hyperparameters. What was the
rationale behind the technique you used? Did you use regularization? Why, or why not? Did you use
an optimization algorithm? Why or why not?
The following task is for Graduate level only (6000 level):
Task 4 (100 points): Create another baseline model (can be any model we covered so far except
a deep learning model). Using the same training data (as above) train your model and evaluate
results using the test set. Compare the results of both models (the Neural Network and the
baseline model). What are the reasons for one model performing better (or not) than the
other? Explain.