Description
Question 0 – Data Prep \[10 points\]
————————————
In this problem set you will train and tune machine learning _regression_ models for a [Superconductivty](https://archive.ics.uci.edu/ml/datasets/Superconductivty+Data) dataset. In this dataset the goal is to predict the critical temperature based on 81 features extracted from a chemical formula of a material. You should use mean squared error (or an equivalent) as the loss function.
In this question you will prepare the data for training and tuning models in the following question. To do so, read the data into Python and create DataFrames or Numpy arrays for the features and dependent regression target (critical temperature).
Then, split the cases into three parts: use 80% of the cases for training, hold 10% for validation and model comparison in question 2, and reserve 10% as a test dataset.
Question 1 – Training and Tuning Models \[70 points\]
—————————————————–
In this question you should train and tune elastic-net, random forest, and gradient boosted decision tree models using the 80% training sample from question 0. Tune hyper-parameters for each model class using 10-fold cross-validation.
### part a
Train a series of elastic net models and choose the mixing parameter `l1_ratio` and the amount of regularization `C` using 10-fold cross-validation over a grid (or grids) of values.
Create a figure or table showing the cross-validated MSE at key points in your grid and identity which hyper-parameters minimize this quantity.
### part b
Train a series of random forest models and use 10-fold cross-validation for hyper-parameter selection. Focus on tuning the tree depth and number of trees. You may, but are not required, to tune other hyper-parameters as well.
Create a figure or table showing the cross-validated MSE for different hyper-parameters considered and identity which hyper-parameters minimize this quantity.
### part c
Train a series of gradient boosted tree models and use 10-fold cross-validation for hyper-parameter selection. Focus on tuning the number of boosting rounds after selecting a suitable learning rate. You may, but are not required, to t une other hyper-parameters as well.
Create a figure or table showing how the cross-validated MSE changes with the the number of boosting rounds and identity which hyper-parameters minimize this quantity.
Question 2 – Validation and Testing \[20 points\]
————————————————-
Using the hyperparameter selected in the previous section, train 3 models – one for each class – on the entire training sample. (You can do this in the previous question for elastic net.)
Use the trained models to make predictions for each case in the validation set created in question 0. Create a nicely formatted table comparing the out-of-sample MSE on the validation dataset for these three models.
Use whichever model performs best in terms of MSE on the validation dataset to make predictions on the test data and report the corresponding MSE.
