Description
10. Under what circumstances might it be beneficial to overfit?
11. Restriction biases of learning algorithms prevent overfitting by restricting the hypothesis
space, while preference biases prevent overfitting by preferring simpler concepts but not
necessarily restricting the hypothesis space. Discuss the pros and cons of preference vs
restriction biases.
12. Person X wishes to evaluate the performance of a learning algorithm on a set of n
examples. X employs the following strategy: Divide the n examples randomly into two
equal-sized disjoint sets, A and B. Then train the algorithm on A and evaluate it on B.
Repeat the previous two steps for N iterations (N large), then average the N performance
measures obtained. Is this sound empirical methodology? Explain why or why not.
13. Prove that an ROC graph must be monotonically increasing.
14. Prove that the ROC graph of a random classifier that ignores attributes and guesses each
class with equal probability must be a diagonal line.