Chapter 2 Defining Random Variables

We write a \(k\)-vector (of scalars) as a row \[ {\boldsymbol{x}}= \begin{bmatrix} x_1 & x_2 & \ldots & x_k \end{bmatrix}. \] The transpose of \({\boldsymbol{x}}\) as \[ {\boldsymbol{x}}^T= \begin{bmatrix} x_1 \\ x_2\\ \vdots \\ x_k \end{bmatrix}. \] We use uppercase letters \(X,Y,Z,\ldots\) to denote random variables. Random vectors are denoted by bold uppercase letters \({\boldsymbol{X}},{\boldsymbol{Y}},{\boldsymbol{Z}},\ldots\), and written as a row vector. For example, \[ {\boldsymbol{X}}= \begin{bmatrix} X_{[1]} & X_{[2]} & \ldots & X_{[k]} \end{bmatrix}. \] In order to distinguish random matrices from vectors, a random matrix is denoted by \({\mathbb{X}}\).

The expectation of \({\boldsymbol{X}}\) is defined as \[ {\mathbb{E}\left[ {\boldsymbol{X}} \right]}= \begin{bmatrix} {\mathbb{E}\left[ X_{[1]} \right]} & {\mathbb{E}\left[ X_{[2]} \right]} & \ldots & {\mathbb{E}\left[ X_{[k]} \right]} \end{bmatrix}. \] The \(k\times k\) covariance matrix of \({\boldsymbol{X}}\) is defined as \[ \begin{aligned} {\mathbb{V}\left[ {\boldsymbol{X}} \right]} &={\mathbb{E}\left[ ({\boldsymbol{X}}-{\mathbb{E}\left[ {\boldsymbol{X}} \right]})^T({\boldsymbol{X}}-{\mathbb{E}\left[ {\boldsymbol{X}} \right]}) \right]} \\ &=\begin{bmatrix} \sigma_1^2 & \sigma_{12} & \ldots & \sigma_{1k} \\ \sigma_{21} & \sigma_{2}^2 & \ldots & \sigma_{2k} \\ \vdots & \vdots & \ddots & \vdots \\ \sigma_{k1} & \sigma_{k2}^2 & \ldots & \sigma_{k}^2 \\ \end{bmatrix}_{k\times k} \end{aligned} \]

where \(\sigma_j={\mathbb{V}\left[ X_{[j]} \right]}\) and \(\sigma_{ij}={\text{Cov}\left[ X_{[i]},X_{[j]} \right]}\) for \(i,j=1,2,\ldots,k\) and \(i\neq j\).

Theorem 2.1 (Linearity of Exectation) Let \({\mathbb{A}}_{l\times k},{\mathbb{B}}_{m\times l}\) be fixed matrices and \({\boldsymbol{c}}\) a fixed vector of size \(l\). If \({\boldsymbol{X}}\) and \({\boldsymbol{Y}}\) are random vectors of size \(k\) and \(m\), respectively, such that \({\mathbb{E}\left[ X \right]}<\infty,{\mathbb{E}\left[ Y \right]}<\infty\), then \[ {\mathbb{E}\left[ {\mathbb{A}}{\boldsymbol{X}}+{\boldsymbol{Y}}{\mathbb{B}}+{\boldsymbol{c}} \right]}={\mathbb{A}}{\mathbb{E}\left[ {\boldsymbol{X}} \right]}+{\mathbb{E}\left[ {\boldsymbol{Y}} \right]}{\mathbb{B}}+{\boldsymbol{c}}. \]