02 ANOVA Homework 1

Lingfeng 2026-09-20

02 ANOVA Homework 1

1. Homework 1

Problem (1)

Consider the one-way fixed-effect ANOVA model $Y_{ij}=\mu_{i}+\varepsilon_{ij},\quad i=1,\dots,r,j=1,\dots,n_{i}$ where $\varepsilon_{ij}\overset{iid}{\sim}N(0,\sigma^{2})$ Device $E(\text{MSE})$ and $E(\text{MSTR})$ .

First, think about the lemma, which is helpful for simplifying the proof: let $\sum_{i=1}^{r}n_{i}=N,n_{i}\in \mathbb{N}$ , $X_{i}\in \mathbb{R}$ , $\bar{X}=\frac{1}{N}\sum_{i=1}^{r}n_{i}X_{i}$ , then $\sum_{i=1}^{r} n_{i}(X_{i}-\bar{X})^{2}=\sum_{i=1}^{r} n_{i}X_{i}^{2}-N \bar{X}^{2}.$
Proof: $\begin{align*}\sum_{i=1}^{r} n_{i}(X_{i}-\bar{X})^{2} & =\sum_{i=1}^{r}n_{i} X_{i}^{2}-2 \bar{X}\sum_{i=1}^{r} n_{i}X_{i }+\bar{X}^{2}\sum_{i=1}^{r} n_{i} \\ &= \sum_{i=1}^{r}n_{i}X_{i}^{2}-2\bar{X}\cdot N\bar{X}+N\bar{X}^{2} \\ &= \sum_{i=1}^{r}n_{i}X_{i}^{2}-N\bar{X}^{2}.\end{align*}$ When $n_{i}=1$ for all $i$ , this reduces to $\sum_{i=1}^{r} (X_{i}-\bar{X})^{2}=\sum_{i=1}^{r} X_{i}^{2}-r\bar{X}^{2}.$

Recall that $E(\text{MSE}) = \frac{1}{N-r}E(\text{SSE}).$ By the lemma applied to each group $i$ , $\begin{align*}\text{SSE} & =\sum_{i=1}^{r}\sum_{j=1}^{n_{i}} (Y_{ij}-\bar{Y}_{i\cdot})^{2} \\ & =\sum_{i=1}^{r}\left( \sum_{j=1}^{n_{i}} Y_{ij}^{2} -n_{i}\bar{Y}_{i\cdot}^{2}\right), \end{align*}$ Since $Y_{ij}\sim N(\mu_{i},\sigma^{2})$ , $E(Y_{ij}^{2})=\sigma^{2}+\mu_{i}^{2}.$ Also $\bar{Y}_{i\cdot}\sim N\left( \mu_{i}, \frac{\sigma^{2}}{n_{i}} \right)$ , then $E(\bar{Y}_{i\cdot}^{2})=\frac{\sigma^{2}}{n_{i}}+\mu_{i}^{2}.$ So $\begin{align*}E(\text{SSE}) & =\sum_{i=1}^{r}(n_{i}\sigma^{2}+n_{i}\mu_{i}^{2}-\sigma^{2}-n_{i}\mu_{i}^{2}) \\ & =\sigma^{2}\sum_{i=1}^{r} (n_{i}-1) \\ & =(N-r)\sigma^{2}.\end{align*}$ Therefore, $E(\text{MSE})=\sigma^{2}$

Recall that $\begin{align*}E(\text{MSTR}) & =\frac{1}{r-1}E(\text{SSTR}) \end{align*}.$ Similarly, $\begin{align*}\text{SSTR} & =\sum_{i=1}^{r} n_{i}(\bar{Y}_{i\cdot}-\bar{Y}_{\cdot \cdot})^{2} \\ & =\sum_{i=1}^{r} n_{i}\bar{Y}_{i\cdot}^{2}-N\bar{Y}^{2}_{\cdot \cdot}.\end{align*}$ We have $\bar{Y}_{i\cdot}\sim N\left( \mu_{i}, \frac{\sigma^{2}}{n_{i}} \right)$ , so $E(\bar{Y}_{i\cdot}^{2})=\frac{\sigma^{2}}{n_{i}}+\mu_{i}^{2}.$ Also, since the group means are independent normal random variables, $E(\bar{Y}_{\cdot \cdot})=\frac{1}{N}\sum_{i=1}^{r}n_{i}\mu_{i}=:\bar{\mu}$ and $\text{Var}(\bar{Y}_{\cdot \cdot})=\frac{1}{N^{2}}\sum_{i=1}^{r} n_{i}^{2} \frac{\sigma^{2}}{n_{i}}=\frac{\sigma^{2}}{N}.$ Hense, $\bar{Y}_{\cdot \cdot}=\frac{1}{N} \sum_{i=1}^{r} n_{i} \bar{Y}_{i\cdot} \sim N\left( \bar{\mu},\ \frac{\sigma^2}{N} \right)$ Then $E(\bar{Y}_{\cdot \cdot}^{2})=\frac{\sigma^{2}}{N}+\bar{\mu}^{2}.$ So $\begin{align*}E(\text{SSTR}) & =\sum_{i=1}^{r} (\sigma^{2}+n_{i}\mu^{2}_{i})-(\sigma^{2}+N\bar{\mu}^{2}) \\ & = (r-1)\sigma^{2}+\left( \sum_{i=1}^{r} n_{i}\mu_{i}^{2}-N\bar{\mu}^{2} \right) \\ & = (r-1)\sigma^{2}+\sum_{i=1}^{r} n_{i}(\mu_{i}-\bar{\mu})^{2}. \end{align*}$ Therefore $E(\text{MSTR})=\sigma^{2}+\frac{1}{r-1}\sum_{i=1}^{r} n_{i}(\mu_{i}-\bar{\mu})^{2}$ where $\bar{\mu}=\frac{1}{N}\sum_{i=1}^{r}n_{i}\mu_{i}.$

Problem (2)

Consider again the one-way fixed-effect ANOVA model $Y_{ij} = \mu_i + \varepsilon_{ij}, \qquad i = 1, \dots, r, \quad j = 1, \dots, n_i,$ where $\varepsilon_{ij} \overset{\text{iid}}{\sim} N(0, \sigma^2).$ Using group $r$ as the reference level, write the model in the linear regression form $\boldsymbol{y} =X\boldsymbol{\beta} + \boldsymbol{\varepsilon}.$

Specify the design matrix $X$ and parameter vector $\boldsymbol{\beta}$ , and derive the ordinary least squares estimates of the model parameters.
Under the linear model formulation, derive the $F$ -statistic for testing $H_0 : \beta_1 = \cdots = \beta_{r-1} = 0,$ and show that it is equivalent to the one-way ANOVA $F$ -statistic.

With group $r$ as the reference level, then $\beta_{0}=\mu_{r}, \quad \beta_{i}=\mu_{i}-\mu_{r},\quad i=1,\dots r-1.$ So $\boldsymbol{\beta}=(\beta_{0},\beta_{1},\dots,\beta_{r-1})^\top$ The design matrix is $X=\begin{pmatrix}\boldsymbol{1}_{n_{1}} & \boldsymbol{1}_{n_{1}} & 0 & \dots & 0 \\ \boldsymbol{1}_{n_{2}} & 0& \boldsymbol{1}_{n_{2}} & \dots & 0 \\ \vdots & \vdots & & \ddots & \vdots \\ \boldsymbol{1}_{n_{r-1}} & 0 & 0 & \dots & \boldsymbol{1}_{n_{r-1}} \\ \boldsymbol{1}_{n_{r}} & 0 & 0 & \dots & 0 \end{pmatrix}$ where $\boldsymbol{1}_{n_{i}}$ denotes an $n_{i}\times 1$ vector of ones.

Notice that $y_{ij}=\begin{cases}\beta_{0}+\beta_{i}+\varepsilon_{ij}, & i=1,\dots,r-1, \\ \beta_{0} +\varepsilon_{ij}, & i=r.\end{cases}$ So using the OLS, $Q(\boldsymbol{\beta})=\sum_{i=1}^{r-1} \sum_{j=1}^{n_{i}} (y_{ij}-\beta_{0}-\beta_{i})^{2}+\sum_{j=1}^{n_{r}} (y_{rj}-\beta_{0})^{2}.$ For $i=1,\dots,r-1$ , let $\begin{align*}\frac{ \partial Q }{ \partial \beta_{i} } & =2 \sum_{j=1}^{n_{i}} (\beta_{i}+\beta_{0}-y_{ij}) \\ & =2n_{i}\beta_{i}+2n_{i}\beta_{0}-2\sum_{j=1}^{n_{i}} y_{ij}=0,\end{align*}$ then $\hat{\beta} _{i}=\bar{y}_{i\cdot}-\hat{\beta}_{0},\quad i=1,\dots,r-1.$ For $i=r$ , let $\begin{align*}\frac{ \partial Q }{ \partial \beta_{0} } & =2\sum_{i=1}^{r-1}\sum_{j=1}^{n_{i}} (\beta_{0}+\beta_{i}-y_{ij})+2\sum_{j=1}^{n_{r}}(\beta_{0}-y_{rj}) \\ & = 2\sum_{i=1}^{r-1}\sum_{j=1}^{n_{i}} (\beta_{0}+\bar{y}_{i\cdot}-\beta_{0}-y_{ij})+2\sum_{j=1}^{n_{r}}(\beta_{0}-y_{rj}) \\ & =2\sum_{j=1}^{n_{r}}(\beta_{0}-y_{rj})=0.\end{align*}$ So $\hat{\beta}_{i}=\begin{cases}\bar{y}_{r\cdot}, & i=0, \\ \bar{y}_{i\cdot}-\bar{y}_{r\cdot}, & i=1,\dots,r-1\end{cases}$

Under $H_{0}:\beta_{1}=\dots=\beta_{r-1}=0$ , $y_{ij}=\beta_{0}+\varepsilon_{ij},$ then $Q_{R}(\beta_{0})=\sum_{i=1}^{r} \sum_{j=1}^{n_{i}} (y_{ij}-\beta_{0})^{2}.$ Let $\frac{ \partial Q_{R} }{ \partial \beta_{0} } =2\sum_{i=1}^{r} \sum_{j=1}^{n_{i}} (\beta_{0}-y_{ij})=0,$ then $\hat{\beta}_{0}=\frac{1}{N}\sum_{i=1}^{r} \sum_{j=1}^{n_{i}} y_{ij}=\bar{y}_{\cdot \cdot}.$ So $\text{SSE}_{R}=\sum_{i=1}^{r} \sum_{j=1}^{n_{i}}(y_{ij}-\bar{y}_{\cdot \cdot})^{2} =\text{SST.}$

From q1, $\hat{y}_{ij}=\begin{cases}\hat{\beta}_{i}+\hat{\beta}_{0}, & i=1,\dots,r-1 \\ \hat{\beta}_{0}, & i=r.\end{cases}$ Derived by $\hat{\beta}_{i}$ , then $\hat{y}_{ij}=\bar{y}_{i\cdot}., \quad i=1,\dots,r$ Thus, $\begin{align*}\text{SSE}_{F} & =\sum_{i=1}^{r} \sum_{j=1}^{n_{i}} (y_{ij}-\hat{y}_{ij})^{2} \\ & = \sum_{i=1}^{r} \sum_{j=1}^{n_{i}} (y_{ij}-\bar{y}_{i\cdot})^{2}=\text{SSE}. \end{align*}$ Therefore, $\text{SSE}_{R}-\text{SSE}_{F}=\text{SST}-\text{SSE}=\text{SSTR}.$
Finally, $F=\frac{(\text{SSE}_{R}-\text{SSE}_{F})/(r-1)}{\text{SSE}_{F} / (N-r)}=\frac{\text{SSTR} / (r-1)}{\text{SSE} / (N-r)}=\frac{\text{MSTR}}{MSE}.$