5.1 - Random Effects

When a treatment (or factor) is a random effect, then we need to re-consider the Null Hypothesis. In a random effects model for the simple case of a single treatment we have:

\(H_0 \colon \sigma_{trt}^{2}= 0 \text{ vs. } H_A \colon \sigma_{trt}^{2}> 0\)

Recall the cell means model for the fixed effect case where we had:

\(Y_{ij}=\mu_i+\epsilon_{ij}\) (Equation 16.2 in the textbook)

In this fixed effect model, \(\mu_i\) are parameters for the treatment means.

For the random effects model we have:

\(Y_{ij}=\mu_i+\epsilon_{ij}\) (Equation 25.1 in the textbook)

but now \(\mu_i\) are \(N(\mu_{.} , \sigma^{2}_{\mu})\), \(\epsilon_{ij}\) are \(N(0 , \sigma^{2})\) and \(\mu_i\) and \(\epsilon_{ij}\) are independent random variables.

The ANOVA calculations and testing procedure is the same as for a fixed effect. We can construct the ANOVA table as before, but now need to show a new column to our ANOVA Table:

Source df SS MS F P EMS (Expected Means Squares)
Trt           \(\sigma_{\epsilon}^{2}+ n\sigma_{\tau}^{2}\)
Error           \(\sigma_{\epsilon}^{2}\)
Total            

We now have listed the Expected Mean Squares (EMS) in the ANOVA table. For the fixed effects models we have discussed so far, this column was not important and so we haven’t been including it. However, for models involving random effects, the EMS column is essential to guide us in constructing appropriate F tests and for computing quantities associated with random effects.

The expected means squares exist for fixed effects as well. We simply have not been showing this column or discussing it. In SAS, when we requested the ‘method=type3’ to get the ordinary ANOVA table output, SAS automatically generated the EMS column.

Rather than focus on the means (as in a fixed effect), we focus here on the variance in the response variable and partition the variance as:

\(\sigma _{Y}^{2}=\sigma _{\tau}^{2}+\sigma _{\epsilon}^{2}\)

\(\text{where } \sigma _{\tau}^{2}=\text{ variance among the treatment means}\)

The terms \(\sigma _{\tau}^{2}\) and \(\sigma _{\epsilon}^{2}\) (estimated as \(s_{\text{among trts}}^{2}\) and \(s_{\text{within trts}}^{2}\) from the data) are referred to as variance components.

Note! Variance components are NOT synonymous with mean squares. This is a common mistake and deserves attention. To calculate variance components, we use the means squares and the EMS for the effect to determine what calculations are required to get the variance component. For example, to get the variance component for the treatment in the single-factor random effects table above, we would need to do the following:

\(s_{\text{among trts}}^{2}= \dfrac{MS_{trt}-MS_{error}}{n}\)

This results from simple algebraic manipulation of the EMS terms shown in the ANOVA table. In models containing many terms, one has to use care to ‘extract’ the variance components. Often the variance components are summed for a total, and then each factor can be expressed as a percent of the total variation in the response variable.

Another common application of variance components is when researchers are interested in the relative size of the treatment effect compared to the within-treatment level variation. This leads to a quantity called the intra-class correlation coefficient or ICC, defined as:

\(ICC=\dfrac{s_{\text{among trts}}^{2}}{s_{\text{among trts}}^{2}+s_{\text{within trts}}^{2}}\)

It can also be thought of as the correlation between the observations within the group. Small values of ICC indicate a large spread of values at each level of the treatment, whereas large values of ICC indicate relatively little spread at each level of the treatment: