13.3 - The Two Factor Mixed Models

Next, consider the case that one of the factors is fixed, say A, and the other one (B) is a random factor. This case is called the two-factor mixed model and the linear statistical model and respective components of variance is

\(y_{ijk}=\mu+\tau_i+\beta_j+(\tau\beta)_{ij}+\varepsilon_{ijk}
\left\{\begin{array}{c}
i=1,2,\ldots,a \\
j=1,2,\ldots,b \\
k=1,2,\ldots,n
\end{array}\right. \)

\(V(\beta_j)=\sigma^2_\beta,\ V((\tau\beta)_{ij})=((a-1)/a)\sigma^2_{\tau\beta},\ V(\varepsilon_{ijk})=\sigma^2\)

\(\sum\limits_{i=1}^a \tau_i=0\)

Here \(\tau_i\) is a fixed effect but \(\beta_j\) and \((\tau \beta)_{ij}\) are assumed to be random effects and \(\epsilon_{ijk}\) is a random error. Furthermore, \(\beta_{j}\) and \(\epsilon_{ijk}\) are NID. The interaction effect is also normal but not independent. There often is a restriction imposed on the interaction which is

\(\sum\limits_{i=1}^a(\tau\beta)_{ij}=(\tau\beta)_{.j}=0 \qquad j=1,2,\ldots,b\)

Because of the sum of interaction effects over the levels of the fixed factor equals zero, this version of the mixed model is called the restricted model. There exists another model which does not include such a restriction and is discussed later. Neither of these models is "correct" or "wrong" - they are both theoretical models for how the data behave. They have different implications for the meanings of the variance components. The restricted model is often used in the ANOVA setting. The unrestricted model is often used for more general designs that include continuous covariates and repeated or spatially correlated measurements.

Once again the tests of hypotheses for the mixed-model are:

\(H_0 \colon \tau_i=0\qquad H_0 \colon \sigma^2_{\beta}=0\qquad H_0 \colon \sigma^2_{\tau\beta}=0\)
\(H_1 \colon \tau_i\neq 0\qquad H_1 \colon \sigma^2_{\beta}>0\qquad H_1 \colon \sigma^2_{\tau\beta}>0\)

Furthermore, test statistics which are based on the expected mean squares are summarized as follows

\(E(MS_A)=\sigma^2+n\sigma^2_{\tau\beta}+\frac{bn\sum\limits_{i=1}^a \tau^2_i}{a-1} \Longrightarrow F_0=\frac{MS_A}{MS_{AB}}\)

\(E(MS_B)=\sigma^2+an\sigma^2_{\beta}\qquad\qquad \Longrightarrow F_0=\frac{MS_B}{MS_E}\)

\(E(MS_{AB})=\sigma^2+n\sigma^2_{\tau\beta}\qquad\qquad \Longrightarrow F_0=\frac{MS_{AB}}{MS_{E}}\)

\(E(MS_E)=\sigma^2\)

In the mixed model, it is possible to estimate the fixed factor effects as before which are shown here:

\(\hat{\mu}=\bar{y}_{..}\)

\(\hat{\tau}_i=\bar{y}_{i..}-\bar{y}_{...}\)

The variance components can be estimated using the analysis of variance method by equating the expected mean squares to their observed values:

\({\hat{\sigma}}^2_{\beta}=\frac{MS_B-MS_E}{an}\)

\({\hat{\sigma}}^2_{\tau\beta}=\frac{MS_{AB}-MS_E}{n}\)

\({\hat{\sigma}}^2=MS_E\)

Example 13.3 is the measurement system capability experiment where here we assume the operator has become a fixed factor while part is left as a random factor. Assuming the restricted version of the mixed effect model, Minitab’s balanced ANOVA routine output is given as follows.

Table 13-6 Analysis of variance (Minitab) for the Mixed Model in Example 13-3

The Restricted Model is Assumed.

Analysis of Variance (Balanced Design)
Factor Type Levels Values
part random 20 1 2 3 4 5 6 7
      8 9 10 11 12 13 14
      15 16 17 18 19 20  
operator random 3 1 2 3        
Analysis of Variance for Y
Source DF SS MS F P
part 19 1185.425 62.391 62.92 0.000
operator 2 2.617 1.308 1.84 0.173
part*operator 38 27.050 0.712 0.72 0.861
Error 60 59.500 0.992    
Total 119 1274.592      
Source Variance Component Error Term Expected mean Square for Each term (using unrestricted model)
1 part 10.2332 4 (4) + 6(1)
2 operator   3 (4) + 2(3) + 40Q[2]
3 part*operator -0.1399 4 (4) + 2(3)
4 Error 0.9917   (4)
Table 13.6 (Design and Analysis of Experiments, Douglas C. Montgomery, 7th Edition)

Like before, there exists a large effect of parts, small operator effect and no part*operator interaction. Notice that again the variance component estimate for the part*operator interaction is negative, which considering its insignificant effect, leads us to assume it is zero and to delete this term from the model.

As mentioned before, there exist alternative analyses for the mixed effect models which are called the unrestricted mixed models. The linear statistical model and components of variance for the unrestricted mixed model are given as:

\(y_{ijk}=\mu+\alpha_i+\gamma_j+(\alpha\gamma)_{ij}+\varepsilon_{ijk}
\left\{\begin{array}{c}
i=1,2,\ldots,a \\
j=1,2,\ldots,b \\
k=1,2,\ldots,n
\end{array}\right. \)

\(V(\gamma_j)=\sigma^2_\beta,\ V((\alpha\gamma)_{ij})=\sigma^2_{\alpha\gamma},\ V(\varepsilon_{ijk})=\sigma^2\)

\(\sum\limits_{i=1}^a \alpha_i=0\)

In the unrestricted mixed model, all of the random terms are assumed to be Normally and independently distributed (NID) and there is not a restriction on the interaction term which was previously imposed. As before, the relevant tests of hypotheses are given by:

\(H_0 \colon \alpha_i=0\qquad H_0 \colon \sigma^2_{\gamma}=0\qquad H_0 \colon \sigma^2_{\alpha\gamma}=0\)
\(H_1 \colon \alpha_i\neq 0\qquad H_1 \colon \sigma^2_{\gamma}>0\qquad H_1 \colon \sigma^2_{\alpha\gamma}>0\)

And the expected mean squares which determine the test statistics are

\(E(MS_A)=\sigma^2+n\sigma^2_{\alpha\gamma}+\frac{bn\sum\limits_{i=1}^a \alpha^2_i}{a-1} \Longrightarrow F_0=\frac{MS_A}{MS_{AB}}\)

\(E(MS_B)=\sigma^2+n\sigma^2_{\alpha\gamma}+an\sigma^2_\gamma\quad \Longrightarrow F_0=\frac{MS_B}{MS_{AB}}\)

\(E(MS_{AB})=\sigma^2+n\sigma^2_{\alpha\gamma}\qquad\qquad \Longrightarrow F_0=\frac{MS_{AB}}{MS_{E}}\)

\(E(MS_E)=\sigma^2\)

Again, to estimate the variance components, the analysis of variance method is used and the expected mean squares are equated to their observed values which result in:

\({\hat{\sigma}}^2_{\gamma}=\frac{MS_B-MS_{AB}}{an}\)

\({\hat{\sigma}}^2_{\alpha\gamma}=\frac{MS_{AB}-MS_E}{n}\)

\({\hat{\sigma}}^2=MS_E\)

Example 13.4 uses the unrestricted mixed model to analyze the measurement systems capability experiment. The Minitab solution for this unrestricted model is given here:

Table 13-7 Analysis of the Experiment in Example 13-3 Using the Unrestricted Model

Analysis of Variance (Balanced Design)
Factor Type Levels Values
part random 20 1 2 3 4 5 6 7
      8 9 10 11 12 13 14
      15 16 17 18 19 20  
operator random 3 1 2 3        
Analysis of Variance for y
Source DF SS MS F P
part 19 1185.425 62.391 87.65 0.000
operator 2 2.617 1.308 1.84 0.173
part*operator 38 27.050 0.712 0.72 0.861
Error 60 59.500 0.992    
Total 119 1274.592      
Source Variance Component Error Term Expected mean Square for Each term (using unrestricted model)
1 part 10.2798 3 (4) + 2(3) + 6(1)
2 operator   3 (4) + 2(3) + Q[2]
3 part*operator -0.1399 4 (4) + 2(3)
4 Error 0.9917   (4)
Table 13.7 (Design and Analysis of Experiments, Douglas C. Montgomery, 7th Edition)

It is difficult to provide guidelines for when the restricted or unrestricted mixed model should be used, because statisticians do not fully agree on this. Fortunately, the inference for the fixed effects does not differ for the 2 factor mixed model which is most often seen, and is usually the same in more complicated models as well.