Description
1. Suppose you have the following data
Observation X1 X2 Class 1 3 1 +1 2 3 -1 +1 3 6 1 +1 4 6 -1 +1 5 1 0 -1 6 0 1 -1 7 0 -1 -1 8 -1 0 -1
(a) [6 points] Find the equation of the hyperplane (in terms of w) WITHOUT solving a quadratic programming (QP) problem. Make a sketch of the problem (i.e., plot the data, unique hyperplane and corresponding dashed lines). (b) [1 point] Calculate the margin. (c) [2 points] Find the α’s of the SVM for classification (again WITHOUT solving a QP problem).
2. In this problem (based on Problem 7 on p. 371), you will use support vector classification in order to predict whether a given car gets high or low gas mileage based on the Auto data set. You can download the data set at https://www-bcf.usc.edu/~gareth/ISL/data.html. It can be helpful to do the lab starting on p. 359 of the textbook first. (a) [1 point] Create a class variable that takes on a “1” for cars with gas mileage above the median, and a “-1” for cars with gas mileage below the median. (b) [5 points] Fit a support vector classifier with linear kernel to the data with various values of cost (i.e., the parameter C from class), in order to predict whether a car gets high or low gas mileage. Report the cross-validation errors associated with different values of this parameter. Comment on your results.
1
(c) [5 points] Now repeat (b), this time using SVMs with radial basis and polynomial kernels, with different values of gamma, degree and cost. Comment on your results. In class, I said the gaussian kernel had the following form: K(Xi,Xj) = exp(−kXi−Xjk2 2/h2). However, in this software package e1071, the radial basis function has the form K(Xi,Xj) = exp(−γkXi −Xjk2 2). Consequently, γ = 1/h2. (d) [1 point] Which kernel function do you prefer and why?
3. (a) [6 points] Make an R program that solves the LS-SVM for nonlinear regression given the parameters γ and bandwidth h for the following gaussian kernel (do not implemenent the cross-validation procedure)
K(Xi,Xj) =
1 √2π exp−kXi −Xjk2 2 2h2 .
(b) [3 points] Try your program on the following function (i.e., experiment with some values of γ and h) X <- seq(0, 1, length.out = 200) y <- (sin(2*pi*(x-0.5)))^2 + rnorm(200, 0, 0.2) Describe the effect when the two tuning parameters γ and h change. Finally, try the values γ = 9.4365 and h = 0.16006. Does this seems a good fit to you?
