6.3 Matrix Notation

Taking the definition of errors from the last section, we can write down a system of \(n\) linear equations: \[ \begin{aligned} Y_1 &= \v{X_1}\v{\beta} + \eps_1 \\ Y_2 &= \v{X_2}\v{\beta} + \eps_2 \\ & \vdots \\ Y_n &= \v{X_n}\v{\beta} + \eps_n \end{aligned} \]

Define \[ \v{Y}=\begin{bmatrix} Y_1 \\ Y_2 \\ \vdots \\ Y_n \end{bmatrix}_{n\times1},\ \mathbb{X}=\begin{bmatrix} 1 & X_{[1]1} & X_{[2]1} & \ldots & X_{[k]1} \\ 1 & X_{[1]2} & X_{[2]2} & \ldots & X_{[k]2} \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 1 & X_{[1]n} & X_{[2]n} & \ldots & X_{[k]n} \end{bmatrix},\mbox{ and } \v{\eps}=\begin{bmatrix} \eps_1 \\ \eps_2 \\ \vdots \\ \eps_n \end{bmatrix}_{n\times1}. \] We can now rewrite the system as the following: \[ \v{Y}=\m{X}\v{\beta}+\v{\eps}. \] We also note that \[ \widehat{\m{Q}}_{\v{XX}}=\sum\limits_{i=1}^n\v{X}^T_i\v{X}_i= \m{X}^T\m{X}, \] and \[ \widehat{\m{Q}}_{\v{X}Y}=\sum\limits_{i=1}^n\v{X}_i^TY_i= \m{X}^T\v{Y}. \] So, we have write the least squares estimator as \[ \widehat{\v{\beta}}=\left[\m{X}^T\m{X}\right]^{-1}\m{X}^T\v{Y}. \] Similarly, the residual vector is \[ \v{e}=\v{Y}-\m{X}\widehat{\v{\beta}}. \] As a consequence of 6.2, we can write \[ \m{X}^T\v{e}=\v{0}. \]