Introduction to Meta-analysis

SPORTSCIENCE · sportsci.org
Perspectives / Research Resources

An Introduction to Meta-analysis

Will G Hopkins

Sportscience 8, 20-21, 2004 (sportsci.org/jour/04/wghmeta.htm)
Sport and Recreation, Auckland University of Technology, Auckland 1020, New Zealand. Email.
Reviewer: George A Kelley, Meta-Analytic Research Group, Community Medicine, West Virginia University, Morgantown, WV 26506-9190.

A meta-analysis is a systematic quantitative review of original research studies of some phenomenon, such as the effect of a specific treatment on some aspect of health or behavior. The meta-analyst expresses the magnitudes of effects from all relevant studies in the same units: percent units are best for most effect representing differences or changes in means; risk, odds and hazard ratios are appropriate for proportions; and ratios are appropriate for counts. The meta-analyst then uses an appropriate weighting factor (the inverse of each effect's error variance) to combine the magnitudes into a mean value and its uncertainty (confidence limits). In a traditional meta-analysis, the true effects are assumed to be homogeneous (have the same value) in the analyzed studies, and some "outlier" studies may be eliminated to satisfy this assumption. In the more recent and realistic random-effect or mixed-model meta-analysis, true values of all effects are assumed to be heterogeneous (different), and the analysis provides an estimate of the heterogeneity as a standard deviation representing unexplained typical true variation in the effect between studies. Inclusion of study and mean subject characteristics in the analysis as covariates may reduce heterogeneity and provide further useful information about the magnitude of the effect in different locations and with different subjects. Published effects are usually larger than their true values, owing to the misuse of statistical significance as a criterion for publication. Several methods are available in the metafor package in R to adjust for such bias.

KEYWORDS: Cochrane Collaboration, funnel plot, meta-regression, mixed model, quantitative analysis, random effect, research, systematic review.

Reprint pdf · Reprint doc · Slideshow

The article/slideshow on the replication crisis also summarizes meta-analysis.

Update April 2024. Extended the last few slides to include detection of outlier studies, fake data, and recent methods to adjust for publication bias.

Update March 2014. Added a forest and moderator plot; mentioned mediator plot.

Update Oct 2013. Substantial make-over to take into account the unified approach to ratios of risks or proportions, odds, hazards and counts in the article on linear models and effect magnitudes. A novel approach of including separate effects for each group from controlled trials or other studies with control, reference or other comparison groups is also described.

Update Aug 2007. Minor improvements to slideshow. See also a more succinct version of the slideshow prepared for but not presented at the 2007 ACSM meeting, as explained in the conference report.

The basis for this article is an updated version of a slideshow accompanying a talk on meta-analysis I presented this year locally and at the University of Bath. The article should meet a need for a straightforward and up-to-date account of meta-analysis suitable for research students and staff in the sport sciences and other biomedical disciplines.

My experience with meta-analysis is limited–one analysis published and three others completed recently–but most of my assertions appear to be consistent with those in the ultimate source of meta-analytic wisdom, the handbook of the Cochrane Collaboration (cochrane.org) I depart from the handbook with my emphasis or novel material on individual responses, standardized differences in means, log transformation, measures of physical performance, and correlations. You will need to refer to the Cochrane handbook for information on topics I don't cover, including survival analysis, intention-to-treat analysis, and meta-analysis of single-subject studies (cases or individual patient data).