Section 3.3.1: General Test Procedures for Studying Random Data

Exercise 25. [HM36] [HM36] Suppose $Y_i = \sum_{j=1}^{n} a_{ij} X_j + \mu_i$ for $1 \le i \le m$, where $X_1, \ldots, X_n$ are independent random variables with mean zero and unit variance, and the matrix $A = (a_{ij})$ has rank $n$.

a) Express the covariance matrix $C = (c_{ij})$, where $c_{ij} = E(Y_i - \mu_i)(Y_j - \mu_j)$, in terms of the matrix $A$.

b) Prove that if $C = (c_{ij})$ is any such matrix, then $\hat{C} = C^t C$, the statistic

$$W = \sum_{i=1}^{m} \sum_{j=1}^{m} (Y_i - \mu_i)(Y_j - \mu_j)\hat{c}_{ij}$$

is equal to $X_1^2 + \cdots + X_n^2$. [Consequently, if the $X_i$ have the normal distribution, $W$ has the chi-square distribution with $n$ degrees of freedom.]

Verified: yes
Solve time: 8m23s

---

Exercise 3.3.1.25 [HM36]

Let

$$ Y_i=\sum_{j=1}^{n}a_{ij}X_j+\mu_i,\qquad 1\le i\le m, $$

where $X_1,\ldots,X_n$ are independent random variables with

$$ E(X_j)=0,\qquad \operatorname{Var}(X_j)=1, $$

and where the $m\times n$ matrix $A=(a_{ij})$ has rank $n$.

Write

$$ X= \begin{pmatrix} X_1\ \vdots\ X_n \end{pmatrix}, \qquad Y-\mu= \begin{pmatrix} Y_1-\mu_1\ \vdots\ Y_m-\mu_m \end{pmatrix}. $$

Then

$$ Y-\mu=AX. $$

(a) Determine the covariance matrix $C$

By definition,

$$ c_{ij}=E\bigl((Y_i-\mu_i)(Y_j-\mu_j)\bigr). $$

Since

$$ Y_i-\mu_i=\sum_{k=1}^{n}a_{ik}X_k, $$

we have

$$ c_{ij}

E!\left( \sum_{k=1}^{n}a_{ik}X_k \sum_{\ell=1}^{n}a_{j\ell}X_\ell \right)

\sum_{k=1}^{n}\sum_{\ell=1}^{n} a_{ik}a_{j\ell}E(X_kX_\ell). $$

Because the $X_r$ are independent with mean $0$ and variance $1$,

$$ E(X_kX_\ell)= \begin{cases} 1,&k=\ell,\ 0,&k\ne \ell. \end{cases} $$

Hence

$$ c_{ij}

\sum_{k=1}^{n}a_{ik}a_{jk}. $$

The $(i,j)$-entry of $AA^t$ is exactly this sum, therefore

$$ \boxed{C=AA^t}. $$

(b) Show that $W=X_1^2+\cdots+X_n^2$

The statement as printed cannot be correct with

$$ \hat C=C^tC, $$

because then

$$ W=(Y-\mu)^t(C^tC)(Y-\mu) =X^t(A^tA)^2X, $$

which is generally not equal to $X^tX$.

The intended meaning is that $\hat C=(\hat c_{ij})$ is a generalized inverse of $C$, satisfying

$$ C\hat C C=C. $$

Since $C=AA^t$ has rank $n$, a convenient choice is the Moore-Penrose inverse

$$ \hat C=C^{+}. $$

We now prove the required identity.

Because $A$ has rank $n$, the matrix $A^tA$ is invertible. Define

$$ \hat C

A(A^tA)^{-2}A^t . $$

Then

$$ C\hat C C

AA^tA(A^tA)^{-2}A^tAA^t

A(A^tA)(A^tA)^{-2}(A^tA)A^t

AA^t

C, $$

so $\hat C$ is indeed a generalized inverse of $C$.

Now compute $W$:

$$ \begin{aligned} W &=(Y-\mu)^t\hat C(Y-\mu)\ &=(AX)^t A(A^tA)^{-2}A^t (AX)\ &= X^t (A^tA) (A^tA)^{-2} (A^tA) X\ &= X^tX. \end{aligned} $$

Therefore

$$ \boxed{ W=X^tX

X_1^2+\cdots+X_n^2 }. $$

Consequently, if $X_1,\ldots,X_n$ are independent $N(0,1)$ random variables, then $W$ is the sum of squares of $n$ independent standard normals. Hence

$$ \boxed{W\sim\chi_n^2}. $$

Thus

$$ \boxed{C=AA^t} $$

and, with $\hat C$ taken as a generalized inverse of $C$,

$$ \boxed{ W=X_1^2+\cdots+X_n^2}. $$