The obvious advantage for performing a meta-analysis is that a large amount of data, pooled across multiple studies, can provide increased precision in addressing the research question. The disadvantage of a meta-analysis is that the studies can be very heterogeneous in their designs, quality, and patient populations and, therefore, it may not be valid to pool them. This issue is something that needs to be evaluated very critically.

Researchers invoke two basic statistical models for meta-analysis, namely, fixed-effects models and random-effects models.

A fixed-effects model is more straightforward to apply, but its underlying assumptions are somewhat restrictive. It assumes that if all the involved studies had tremendously large sample sizes, then they all would yield the same result. In essence, a fixed-effects model assumes that there is no inter-study variability (study heterogeneity). This statistical model accounts only for intra-study variability.

A random-effects model, however, assumes that the eligible studies actually represent a random sample from a population of studies that address the research question. It accounts for intra-study and inter-study variability. Thus, a random-effects model tends to yield a more conservative result, i.e., wider confidence intervals and less statistical significance than a fixed-effects model.

A random-effects model is more appealing from a theoretical perspective, but it may not be necessary if there is very low study heterogeneity. A formal test of study heterogeneity is available. Its results, however, should not determine whether to apply a fixed-effects model or random-effects model. You need to use your own judgment as to which model should be applied.

The test for study heterogeneity is very powerful and sensitive when the number of studies is large. It is very weak and insensitive if the number of studies is small. Graphical displays provide much better information as to the nature of study heterogeneity. Some medical journals require that the authors provide the test of heterogeneity, along with a fixed-effects analysis and a random-effects analysis.