10.3 - Other Covariance Structures

Variance Components (VC)

\(\begin{bmatrix} \sigma_1^2 & \cdots & \cdots & \vdots\\ \vdots& \sigma_1^2 & \cdots & \vdots\\ \vdots & \cdots & \sigma_1^2 & \vdots\\ \cdots & \cdots & \cdots & \sigma_1^2 \end{bmatrix}\)

The variance component structure (VC) is the simplest, where the correlation of errors within a subject are presumed to be 0. This structure is the default setting in Proc Mixed, but is not a reasonable choice for repeated measures designs. I sometimes include it in the exploration process to get a sense of the effect of fitting other structures.

Compound Symmetry

\(\sigma^2 \begin{bmatrix} 1.0 & \rho & \rho & \rho \\ & 1.0 & \rho & \rho \\ & & 1.0 & \rho \\ & & & 1.0 \end{bmatrix} = \begin{bmatrix} \sigma_b^2+\sigma_e^2 & \sigma_b^2 & \sigma_b^2 & \sigma_b^2 \\ & \sigma_b^2+\sigma_e^2 & \sigma_b^2 & \sigma_b^2 \\ & & \sigma_b^2+\sigma_e^2 & \sigma_b^2 \\ & & & \sigma_b^2+\sigma_e^2 \end{bmatrix}\)

The simplest covariance structure that includes within-subject correlated errors is compound symmetry (CS). Here we see correlated errors between time points within subjects, and note that these correlations are presumed to be the same for each set of times, regardless of how distant in time the repeated measures are made.

First Order Autoregressive AR(1)

\(\sigma^2 \begin{bmatrix} 1.0 & \rho & \rho^2 & \rho^3 \\ & 1.0 & \rho & \rho^2 \\ & & 1.0 & \rho \\ & & & 1.0 \end{bmatrix}\)

The autoregressive (Lag 1) structure considers correlations to be highest for time adjacent times, and a systematically decreasing correlation with increasing distance between time points. For one subject, the error correlation between time 1 and time 2 would be \(\rho^{t_1-t_2}\) . Between time 1 and time 3 the correlation would be less, and equal to \(\rho^{t_1-t_3}\). Between time 1 and 4, the correlation is less yet, as \(\rho^{t_1-t_4}\), and so on. Note, however, that this structure is only applicable for evenly spaced time intervals for the repeated measure.

Spatial Power

\(\sigma^2 \begin{bmatrix} 1.0 & \rho^{\frac{|t_1-t_2|}{|t_1-t_2|}} & \rho^{\frac{|t_1-t_3|}{|t_1-t_2|}} & \rho^{\frac{|t_1-t_4|}{|t_1-t_2|}} \\ & 1.0 & \rho^{\frac{|t_2-t_3|}{|t_1-t_2|}} & \rho^{\frac{|t_2-t_4|}{|t_1-t_2|}} \\ & & 1.0 & \rho^{\frac{|t_3-t_4|}{|t_1-t_2|}} \\ & & & 1.0 \end{bmatrix}\)

When time intervals are not evenly spaced, a covariance structure equivalent to the AR(1) is the spatial power (SP(POW)). The concept is the same as the AR(1) but instead of raising the correlation to powers of 1, 2,, 3, … , the correlation coefficient is raised to a power that is actual difference in times (e.g. \(|t_1-t_2|\) for the correlation between time 1 and time 2). This method requires having a quantitative expression of the times in the data so that it can be specified for calculation of the exponents in the SP(POW) structure. If an analysis is run wherein the repeated measures are equally spaced in time, the AR(1) and SP(POW) structures yield identical results.

Unstructured Covariance

\( \begin{bmatrix} \sigma_1^2 & \sigma_{12} & \sigma_{13} & \sigma_{14} \\ & \sigma_2^2 & \sigma_{23} & \sigma_{24} \\ & & \sigma_3^2 & \sigma_{34}\\ & & & \sigma_4^2 \end{bmatrix}\)

The Unstructured covariance structure (UN) is the most complex because it is estimating unique correlations for each pair of time points. It is not uncommon to find out that you are not able to use this structure. SAS will return an error message indicating that there are too many parameters to estimate with the data.

Choosing the Best Covariance Structure

Here is an excerpt from the SAS Manual, Mixed Models Analyses Using the SAS System Course Notes, that explains how to approach this:

  Unfortunately, our attempt to share a very RECENT perspective by a relatively small number of statistics and statistics related research has somewhat sidetracked the focus of lesson 1. Would like to attempt to provide some clarity to some of the discussion on the discussion forum about the bar chart vs. interval charts.

You can use information criteria produced by the MIXED procedure as a tool to help you select the model with the most appropriate covariance structure. The smaller the information criteria value is, the better the model is. Theoretically, the smaller the -2 Res Log Likelihood is, the better the model is. However, you can always make this value smaller by adding parameters to the model. Information criteria attached penalties to the negative -2 Res Log Likelihood value; that is, the more the parameters, the bigger the penalties.

Two commonly used information criteria are Akaike's (1974) and Schwartz's (1978). Generally speaking, BIC tends to choose less complex models than AIC. Because choosing a model that is too simple inflates Type I error rate, when Type I error control is the highest priority, you may want to use AIC. On the other hand, if loss of power is more of a concern, BIC might be preferable (Guerin and Stroup 2000).

Starting in the Release 8.1, the MIXED procedure produces another information criteria, AICC. AICC is a finite-sample corrected Akaike Information Criterion. For small samples, it reduces the bias produced by AIC; for large samples, AICC converges to AIC. In general, AICC is preferred to AIC. For more information on information criteria, especially AICC, referr to Burnham, K. P. and Anderson, D. R. (1998).

The basic idea for repeated measures analysis is that, among plausible within-subject covariance models given a particular study, the model that minimizes AICC or BIC (your choice) is preferable. When AICC or BIC are close, the simpler model is generally preferred.