16.8 - Random Effects / Sensitivity Analysis

A corresponding linear model for the random-effects approach is as follows:

\(Y_k = \theta + t_k + e_k\)

where \(Y_k , \theta\), and \(e_k\) are the same as described above and \(t_k\) is a random effect for the \(k^{th}\) study.

It is assumed that \(t_1 , t_2 , \dots , t_k\) are independent and identically distributed as \(N \left(0 , \omega^{2} \right)\) random variables. The variance term \(\omega^2\) reflects inter-study variability.

Whereas \(\text{Var}\left(Y_k\right) = \sigma_k^2\) is the variance associated with the fixed-effects linear model,

\(\text{Var}\left(Y_k\right) = \sigma_k^2+\omega^2\) is the variance associated with the random-effects linear model.

A weighted analysis will be applied, analogous to the weighted analysis for the fixed-effects linear model, but the weights are different. The overall weighted treatment effect is:

\(Y=\left( \sum_{k=1}^{K}w_{k}^{*}Y_k \right) / \left( \sum_{k=1}^{K}w_{k}^{*} \right) \)

where

\(w_{k}^{*}=1/ (S_{k}^{2}+\hat{\omega}^2)\)

\(\hat{\omega}^2 = max(0, Q-K+1) \left( \sum_{k=1}^{K}w_{k} \right) / \left( \left( \sum_{k=1}^{K}w_{k} \right)^2 - \sum_{k=1}^{K}w_{k}^2 \right) \)

and where Q is the heterogeneity statistic and \(w_k\) is the weight for the \(k^{th}\) study, which were defined previously for the weighted analysis in the fixed-effects linear model.

Analogous to the fixed-effects linear model, the variance of Y in the random-effects linear model is:

\(S^2= \dfrac{1}{\left(\sum_{k=1}^{K}w_k^* \right)}\)

and statistical inference is performed in a similar manner.

If there exists a large amount of study heterogeneity, then \(\hat{\omega}^2\) will be very large and will dominate in the expression for the weight in the \(k^{th}\) study, i.e.,

\(w_k^*= \dfrac{1}{\left(S_k^2 + \hat{\omega}^2\right)} \approx \dfrac{1}{\hat{\omega}^2} \)

Therefore, in such a situation with a relatively large amount of heterogeneity, the weight for each study will approximately be the same and the weighted analysis for the random-effects linear model will approximate an unweighted analysis.

SAS program P21.1 in this module illustrates the fixed and random effects approaches to the asthma meta-analysis. Try it!

Graphs of the Treatment Differences Section

Graphical displays showing the estimated treatment difference and its confidence interval for every study are very useful for evaluating treatment effects over time or with respect to other factors. An example is given below:

x x x -10 10 μ1 - μ0 x x 1993 1997 1998 2001 2002

Sensitivity Analyses Section

Statistical diagnostics (sensitivity analyses) should be performed to investigate the validity and robustness of the meta-analysis via applying the meta-analytic approach to subsets of the K studies, and/or applying the leave-one-out method.

The steps for the leave-one-out method are as follows:

  • Remove the first of the K studies and conduct the meta-analysis on the remaining K - 1 studies
  • Remove the second of the K studies and conduct the meta-analysis on the remaining K - 1 studies
  • Continue this process until there are K distinct meta-analyses (each with K - 1 studies)

If the results of the K meta-analyses in the leave-one-out method are consistent, then there is confidence that the overall meta-analysis is robust. The likelihood of consistency increases as K increases. The idea here is that removing one of the studies from the meta-analysis should not affect the overall results. If this does occur, this suggests that there exists a lack of homogeniety among the studies involved.

Rather than invoke the leave-one-out method, researchers often prefer to perform a sensitivity analysis by applying the meta-analysis to subsets of studies based on high-quality versus low-quality studies, randomized versus non-randomized studies, early studies versus late studies, etc.