This is a question that comes from An Introduction to Statistical Learning.

Suppose that you collect a set of data (n = 100 observations) containing a single predictor and a quantitative response. You then fit two models to this data:

$$ \begin{align} Y &= \beta_{0} + \beta_{1} X + \epsilon \\ Y &= \beta_{0} + \beta_{1} X + \beta_{2} X^2 + \beta_{3} X^3 + \epsilon \end{align} $$

1. Suppose that the true relationship between $X$ and $Y$ is linear, i.e. $Y = \beta_{0} + \beta_{1} X + \epsilon$. Consider the training residual sum of squares (RSS) for the two models. Do you expect one to be lower than the other, about the same, or is there not enough informatino to tell? Justify your answer.

2. Answer the first question, but instead of the training RSS, consider the test RSS.

3. Suppose that the true relationship between $X$ and $Y$ is not linear, but we don't know "how far from linear" it is. That means, it might be quadratic, cubic, or something else. Consider the training RSS for the two models. Do you expect one to be lower than the other, about the same, or is there not enough information to tell? Justify your answer.

4. Consider the test RSS for the two models in the previous question. Do you expect one to be lower than the other, about the same, or is there not enough information to tell? Justify your answer.