13.1 - Publication Bias

Published reports should inform the reader about all aspects of study design, conduct, analysis, and interpretation that are relevant for assessing the internal and external validity of the trial. The content and quality of trial reports in the literature, however, remain inconsistent on these points - this process is not perfect. The content of the medical literature reflects an imperfect editorial and peer review process. Despite the limitations of the peer review process, good alternatives currently do not exist for judging the merits of scientific papers.

Publication bias is the tendency for studies with “positive” findings to be preferentially selected for publication over those with “negative” findings, (i.e., it did not find a statistically significant result). If an editor has a choice of publishing a positive study and one with negative results, they may prefer publishing the positive results for various reasons. However, negative studies are very important and should be made known as well. For instance, early stopping a study of interferon gamma-1b when an interim analysis showed that patients with IPF did not benefit from the treatment is important imformation for other IPF patients who may have been prescribed interferon gamma-1b in off-label use and for others taking it for the conditions for which it already has approval (FDA Public Health Advisory on Interferon gamma-1b).

If the published literature is used as a basis for drawing conclusions about a treatment or a group of related treatments from independent studies, then an impression biased in favor of the treatment could result. Editors and referees are not the only ones to blame for the presence of publication bias. Investigators tend to lose enthusiasm for negative results because they may be viewed as less glamorous and even viewed as failures. This could lead to weaker reports, or even no reports, being submitted for publication. Journal editors and referees can reduce publication bias in two ways. First, they should assign greater weight to methodologic rigor and thorough reporting than to statistical significance. Second, they must be willing to report negative findings from sound studies with as much enthusiasm as positive reports.