6.3 - Testing Random Effects

Random effects can appear in both factorial and nested designs. By inspecting the EMS quantities, we can determine the appropriate F-statistic denominator for a given source. Let us look at two-factor studies.

 

Factorial Design

Recall the Greenhouse example in Section 5.1. In this example, there were two crossed factors (fert and species). We treated both factors to be fixed and the SAS  proc mixed ANOVA table was as follows:

 
Type 3 Analysis of Variance
Source DF Sum of Squares Mean Square Expected Mean Square Error Term Error DF F Value Pr > F
fert 3 745.437500 248.479167 Var(Residual) + Q(fert,fert*species) MS(Residual) 40 73.10 <.0001
species 1 236.740833 236.740833 Var(Residual) + Q(species,fert*species) MS(Residual) 40 69.65 <.0001
fert*species 3 50.584167 16.861389 Var(Residual) + Q(fert*species) MS(Residual) 40 4.96 0.0051
Residual 40 135.970000 3.399250 Var(Residual) . . . .

 

If we inspect the EMS quantities in the output, we see when both factors are fixed in the 2-factor crossed study that the correct denominator for all F-tests is Error Mean Squares. 

Now let us consider a case in which both factors A and B are random effects in the factorial design (i.e. factors A and B crossed, and both are random effects). The expected mean squares for each of the source of variations in the ANOVA model would be as follows:

Source EMS
A \(\sigma^2+nb\sigma_{\alpha}^{2}+n\sigma_{\alpha\beta}^{2}\)
B \(\sigma^2+na\sigma_{\beta}^{2}+n\sigma_{\alpha\beta}^{2}\)
A × B \(\sigma^2+n\sigma_{\alpha\beta}^{2}\)
Error \(\sigma^2\)
Total  

The F-tests following from the EMS above would be:

Source EMS F
A \(\sigma^2+nb\sigma_{\alpha}^{2}+n\sigma_{\alpha\beta}^{2}\) MSA / MSAB
B \(\sigma^2+na\sigma_{\beta}^{2}+n\sigma_{\alpha\beta}^{2}\) MSB / MSAB
A × B \(\sigma^2+n\sigma_{\alpha\beta}^{2}\) MSAB / MSE
Error \(\sigma^2\)  
Total    

Here we can see the ramifications of having random effects. In fixed-effects models, the denominator for the F-statistics in significance testing was the mean square error (MSE). In random-effects models, however, we may have to choose different denominators depending on the term we are testing.

In general, the F-statistic for testing the significance of a given effect is the ratio of two MS values, with MS of the effect as the numerator and the denominator MS is chosen such that the F-statistic equals 1 if \(H_0\) is true and is greater than 1 if \(H_a\) is true.

Following this logic, we can see that when testing for the interaction effect of 2 random factors, the correct denominator is the error mean squares. Therefore the test statistic for testing \(A \times B\) is \(\frac{MSAB}{MSE}\). However, when we are testing for the main effect of factor A, the correct denominator would be \(MSAB\). 

Recall that the EMS quantities are the population counterparts for the MS estimates which actually are sample statistics. Examination of EMS expressions can therefore be used to choose the correct denominator for the F-statistic utilized for testing significance and will be discussed in detail in Section 6.7.

 

Nested Design

In the case of a nested design, where factor B is nested within the levels of factor A and both are random effects, the expected mean squares for each of the source of variations in the ANOVA model would be as follows:

Source EMS
A \(\sigma^2+bn\sigma_{\alpha}^{2}+n\sigma_{\beta}^{2}\)
B(A) \(\sigma^2+n\sigma_{\beta}^{2}\)
Error \(\sigma^2\)
Total  

The F-tests follow from the EMS above:

Source EMS F
A \(\sigma^2+bn\sigma_{\alpha}^{2}+n\sigma_{\beta}^{2}\) MSA / MSB(A)
B(A) \(\sigma^2+n\sigma_{\beta}^{2}\) MSB(A) / MSE
Error \(\sigma^2\)  
Total    

 

Special Case: Fully Nested Random Effects Design

Here, we consider a special case of random effects models where each factor is nested within the levels of the next ‘order’ of a hierarchy. This Fully Nested Random Effects model is similar to Russian Babushka dolls where the smaller dolls are nested within the next larger one.

Consider 3 random factors A, B, and C that are hierarchically nested. That is C is nested in (B, A) combinations and B is nested within levels of A. Suppose there are n observations made at the lowest level.

The statistical model for this case is:

\(Y_{ijkl}=\mu+\alpha_i+\beta_{j(i)}+\gamma_{k(ij)}+\epsilon_{ijkl}\)

where \(i = 1, 2, \dots, a\), \(j = 1, 2, \dots, b\), \(k = 1, 2, \dots, c\) and \(l = 1, 2, \dots, n\).

We will also have \(\epsilon_{ijkl} \overset{iid}{\sim} \mathcal{N}\left(0, \sigma^2\right)\), \(\gamma_{k(ij)} \overset{iid}{\sim} \mathcal{N}\left(0, \sigma^2_{\gamma}\right)\), \(\beta_{i(j)} \overset{iid}{\sim} \mathcal{N}\left(0, \sigma^2_{\beta}\right)\) and \(\alpha_{i} \overset{iid}{\sim} \mathcal{N}\left(0, \sigma^2_{\alpha}\right)\). 

The dfs and expected mean squares for this design would be as follows:

Source DF EMS F
A (a-1) \(\sigma_{\epsilon}^{2}+n\sigma_{\gamma}^{2}+nc\sigma_{\beta}^{2}+ncb\sigma_{\alpha}^{2}\) MSA / MSB(A)
B(A) a(b-1) \(\sigma_{\epsilon}^{2}+n\sigma_{\gamma}^{2}+nc\sigma_{\beta}^{2}\) MSB(A) / MSC(AB)
C(A,B) ab(c-1) \(\sigma_{\epsilon}^{2}+n\sigma_{\gamma}^{2}\) MSC(AB) / MSE
Error abc(n-1) \(\sigma_{\epsilon}^{2}\)  
Total abcn -1    

In this case, each F-test we construct for the sources will be based on different denominators.