Meta-analysis: Fact or fiction? How to interpret meta-analyses

Abstract

Objectives. Widespread use of increasingly complex statistical methods makes it ever more challenging to adequately assess the results reported and conclusions drawn in meta-analytic research. This paper aims to identify potential fallacies by in-depth examination of recent publications on mood disorders. Methods. Three meta-analyses were selected based on availability of data and representativeness of methods employed. By means of detailed re-analysis, several widespread methodological problems were identified, and the example data were used to illustrate and discuss them. Results. General points addressed include clear formulation of the research question, choice of effect size measures, and general choice of model. Data quality problems like missing data and publication bias are discussed along with methods to deal with them. Furthermore, aspects of meta-analytic modelling like the use of fixed or random effects, data aggregation, as well as the use of subgroups are explained, and issues of excessive complexity and data dredging pointed out. Finally, the benefit of diagnostic tools like confidence bands and the importance of transparency regarding data and methodology for the interpretation of meta-analytic results are highlighted. Conclusions. Practically relevant quality criteria for readers to bear in mind when dealing with meta-analytic publications are summarized in a ten point checklist.

Key words::

• Depression
• antidepressants
• major depressive disorder
• metaanalysis
• interpretation

Acknowledgements

WH and KK conducted the statistical analysis and wrote the first draft of the paper. All authors contributed to the final draft. WH and KK contributed equally to this paper. This research received no specific grant from any funding agency in the public, commercial, or non-profit sectors.

Statement of Interest

WH and KK have no conflict of interest with any commercial or other associations in connection with the submitted article. GP has been employed by Lundbeck from 2005 to 2007, and has received educational grants from Lundbeck and Servier. He has worked as a consultant for AstraZeneca, Bristol-Myers Squibb, GlaxoSmithKline, Lundbeck and Servier; and has served on speakers’ bureaus for AstraZeneca, Bristol-Myers Squibb, Lundbeck, MSD, Nycomed, and Servier. MEF has received a travel grant from Eli Lilly. PF has no conflict of interest with any commercial or other associations in connection with the submitted article. SK has received grant/research support from Eli Lilly, Lundbeck, Bristol-Myers Squibb, GlaxoSmithKline, Organon, Sepracor, and Servier; has served as a consultant or on advisory boards for AstraZeneca, Bristol-Myers Squibb, GlaxoSmithKline, Eli Lilly, Lundbeck, MSD, Pfizer, Organon, Schwabe, Sepracor, Servier, Janssen, and Novartis; and has served on speakers’ bureaus for AstraZeneca, BMS, Angelini, Eli Lily, Lundbeck, Schwabe, Sepracor, Servier, Pfizer, Pierre Fabre, and Janssen.