6.5 - Introduction to Mixed Models
6.5 - Introduction to Mixed ModelsTreatment designs can be comprised of both fixed and random effects. When we have this situation, the treatment design is referred to as a mixed model. Mixed models are by far the most commonly encountered treatment designs. The three possible situations we now have are often referred to as Model I (fixed effects only), Model II (random effects only), and Model III (mixed) ANOVAs. When designating the effects of a mixed model as fixed or random, the following rule will be useful.
Below are the ANOVA layouts of two basic mixed models with 2-factors.
Factorial
In the simplest case of a balanced mixed model in a factorial design, we may have two factors, A and B, in which factor A is a fixed effect and factor B is a random effect. The statistical model is similar to what we have seen before:
\(y_{ijk} = \mu + \alpha_i + \beta_j + (\alpha\beta)_{ij} + \epsilon_{ijk}\) where \(i=1,2,\dots, a\), \(j=1,2,\dots, b\) and \(k=1,2, \dots, n\).
Here, \(\sum_{i} \alpha_i = 0, \) \(\beta_j \sim \mathcal{N}\left(0, \sigma^2_{\beta}\right), \) \((\alpha\beta)_{i,j} \sim N(0,\sigma^2_{\alpha\beta}) \) and \(\epsilon_{ijk} \sim \mathcal{N}\left(0, \sigma^2\right)\). Also, \(\beta_j, (\alpha\beta)_{ij}\) and \(\epsilon_{ij}\) are pairwise independent.
In this case, we have the following ANOVA:
Source | DF | EMS |
|---|---|---|
A | (a-1) | \(\sigma^2+n\sigma_{\alpha\beta}^{2}+nb\frac{\sum\alpha_{i}^{2}}{a-1}\) |
B | (b-1) | \(\sigma^2+n\sigma_{\alpha\beta}^{2}+na\sigma_{\beta}^{2}\) |
A \(\times\) B | (a-1)(b-1) | \(\sigma^2+n\sigma_{\alpha\beta}^{2}\) |
Error | ab(n-1) | \(\sigma^2\) |
Total | abn-1 |
The F-tests are set up based on the EMS column above and we can see that we have to use different denominators in testing significance for the various sources in the ANOVA table:
Source | EMS | F |
|---|---|---|
A | \(\sigma^2+n\sigma_{\alpha\beta}^{2}+nb\frac{\sum\alpha_{i}^{2}}{a-1}\) | MSA / MSAB |
B | \(\sigma^2+n\sigma_{\alpha\beta}^{2}+na\sigma_{\beta}^{2}\) | MSB / MSAB |
A × B | \(\sigma^2+n\sigma_{\alpha\beta}^{2}\) | MSAB / MSE |
Error | \(\sigma^2\) | |
Total |
As a reminder, the null hypothesis for a fixed effect is that the \(\alpha_i\)'s are equal, whereas the null hypothesis for the random effect is that the \(\sigma_{\beta}^{2}\) is equal to zero.
It is important to mention that the mixed model presented here is known as the “unrestricted model”, which is the model utilized in SAS as well. However, there is another “restricted” version of this model which is used in our course text. In practice, the most important difference between these models is how the F-stat for the random effect is calculated. More specifically, for the unrestricted model here it is MSB/MSAB, whereas for the restricted model, it is MSB/MSE. You can read more about the differences between these models in STAT 503 Section 13.3.
Nested
In the case of a balanced nested treatment design, where A is a fixed effect and B(A) is a random effect, the statistical model would be:
\(y_{ijk} = \mu + \alpha_i + \beta_{j(i)} + \epsilon_{ijk}\) where \(i=1,2,\dots, a\), \(j=1,2,\dots, b\) and \(k=1,2, \dots, n\).
Here, \(\sum_{i} \alpha_i = 0\), \(\beta_{j(i)} \sim \mathcal{N}\left(0, \sigma^2_{\beta(\alpha)}\right)\) and \(\epsilon_{ijk} \sim \mathcal{N}\left(0, \sigma^2\right)\).
We have the following ANOVA for this model:
Source | DF | EMS |
|---|---|---|
A | (a-1) | \(\sigma_{\epsilon}^{2}+n\sigma_{\beta(\alpha)}^{2}+bn\frac{\sum\alpha_{i}^{2}}{a-1}\) |
B(A) | a(b-1) | \(\sigma_{\epsilon}^{2}+n\sigma_{\beta(\alpha)}^{2}\) |
Error | ab(n-1) | \(\sigma_{\epsilon}^{2}\) |
Total | abn-1 |
Here is the same table with the F-statistics added. Note that the denominators for the F-test are different.
Source | EMS | F |
|---|---|---|
A | \(\sigma_{\epsilon}^{2}+n\sigma_{\beta(\alpha)}^{2}+bn\frac{\sum\alpha_{i}^{2}}{a-1}\) | MSA / MSB(A) |
B(A) | \(\sigma_{\epsilon}^{2}+n\sigma_{\beta(\alpha)}^{2}\) | MSB(A) / MSE |
Error | \(\sigma_{\epsilon}^{2}\) | |
Total |
F-Calculation Facts
As can be seen from the examples above (and in the previous sections), when significance testing in random or mixed models the denominator of the F-statistic is no longer always the MSE value and has to be aptly chosen. Recall that the F-statistic for testing the significance of a given effect is the ratio with the numerator equal to the MS value of the effect, and the denominator is also an MS value of an effect included in the ANOVA model. Furthermore, it can be said the F-statistic has a non-central distribution when \(H_a\) is true and a central F-distribution when \(H_0\) is true.
When \(H_a\) is true, the non-centrality parameter of the non-central F-distribution depends on the type of effect (fixed vs random) and equals \(\sum_{i=1}^T {\alpha_{i}^2}\) for a fixed effect and \(\sigma_{trt}^2\) for a random effect. Here \(\alpha_i=\mu_i-\mu\), where \(\mu_i\) is the \(i^{th}\) level of the fixed effect for \((i=1,2,...,T)\), \(\mu\) is the overall mean, and \(\sigma_{trt}^2\) is the variance component associated with the random effect. Also, the MS under true \(H_a\) equals to MS under true \(H_0\) plus non-centrality parameter, so that
F-statistic = \(\dfrac{\text{MS when \(H_0\) is true + non-centrality parameter}}{\text{MS when \(H_0\) is true}}\).
The above identity can be used to identify the correct denominator (also called the error term) with the aid of EMS expressions displayed in the ANOVA table. This can be summarized by the following rule.