6.2 Errors and Residuals

Recall that \(\v{\beta}\) denotes the coefficients of the best linear predictor ??. We first define the fitted value as \[ \widehat{Y}_i=\v{X}_i\widehat{\v{\beta}}\mbox{ for } i=1,2,\ldots,n. \] For the least squares estimators, we define the errors and residuals in the following way: \[ \eps_i=Y_i-\v{X}_i\v{\beta}, \mbox{ and } e_i=Y_i-\widehat{Y}_i. \]

Theorem 6.2 (Least Squares Error) If  \(\widehat{\mathbb{Q}}_{\v{XX}}\) is non-singular, then
1. \(\sum\limits_{i=1}^n\v{X}_i^Te_i=\v{0}\)
2. \(\sum\limits_{i=1}^ne_i=0\)

Proof. \[ \begin{aligned} \sum\limits_{i=1}^n\v{X}_i^Te_i &=\sum\limits_{i=1}^n\v{X}_i^T(Y_i-\widehat{Y}_i) \\ &=\sum\limits_{i=1}^n\v{X}_i^TY_i-\sum\limits_{i=1}^n\v{X}_i^T\widehat{Y}_i \\ &=\sum\limits_{i=1}^n\v{X}_i^TY_i-\sum\limits_{i=1}^n\v{X}_i^T\v{X}_i\v{\widehat{\beta}} \\ &=\widehat{\m{Q}}_{\v{X}Y}-\widehat{\m{Q}}_{\v{XX}}\v{\widehat{\beta}} \\ &=\widehat{\m{Q}}_{\v{X}Y}-\widehat{\m{Q}}_{\v{XX}} \left( \widehat{\m{Q}}_{\v{XX}}^{-1} \widehat{\m{Q}}_{\v{X}Y} \right) \\ &=\v{0} \end{aligned} \] From the first row of (1) we get

\[ \sum\limits_{i=1}^n e_i=0. \] Hence the result.