0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Original Investigation |

Association Between Analytic Strategy and Estimates of Treatment Outcomes in Meta-analyses

Agnes Dechartres, MD, PhD1,2,3; Douglas G. Altman, DSc4; Ludovic Trinquart, PhD1,5; Isabelle Boutron, MD, PhD1,2,3,6; Philippe Ravaud, MD, PhD1,2,3,5,6
[+] Author Affiliations
1Centre de Recherche Epidémiologie et Statistique, INSERM U1153, Paris, France
2Centre d’Épidémiologie Clinique, Hôpital Hôtel Dieu, Assistance Publique des Hôpitaux de Paris, Paris, France
3Faculté de Médecine, Université Paris Descartes, Sorbonne Paris Cité, Paris, France
4Centre for Statistics in Medicine, Oxford, United Kingdom
5Department of Epidemiology, Mailman School of Public Health, Columbia University, New York, New York
6French Cochrane Centre, Paris, France
JAMA. 2014;312(6):623-630. doi:10.1001/jama.2014.8166.
Text Size: A A A
Published online

Importance  A persistent dilemma when performing meta-analyses is whether all available trials should be included in the meta-analysis.

Objectives  To compare treatment outcomes estimated by meta-analysis of all trials and several alternative analytic strategies: single most precise trial (ie, trial with the narrowest confidence interval), meta-analysis restricted to the 25% largest trials, limit meta-analysis (a meta-analysis model adjusted for small-study effect), and meta-analysis restricted to trials at low overall risk of bias.

Data Sources  One hundred sixty-three meta-analyses published between 2008 and 2010 in high-impact-factor journals and between 2011 and 2013 in the Cochrane Database of Systematic Reviews: 92 (705 randomized clinical trials [RCTs]) with subjective outcomes and 71 (535 RCTs) with objective outcomes.

Data Synthesis  For each meta-analysis, the difference in treatment outcomes between meta-analysis of all trials and each alternative strategy, expressed as a ratio of odds ratios (ROR), was assessed considering the dependency between strategies. A difference greater than 30% was considered substantial. RORs were combined by random-effects meta-analysis models to obtain an average difference across the sample. An ROR greater than 1 indicates larger treatment outcomes with meta-analysis of all trials. Subjective and objective outcomes were analyzed separately.

Results  Treatment outcomes were larger in the meta-analysis of all trials than in the single most precise trial (combined ROR, 1.13 [95% CI, 1.07-1.19]) for subjective outcomes and 1.03 (95% CI, 1.01-1.05) for objective outcomes). The difference in treatment outcomes between these strategies was substantial in 47 of 92 (51%) meta-analyses of subjective outcomes (meta-analysis of all trials showing larger outcomes in 40/47) and in 28 of 71 (39%) meta-analyses of objective outcomes (meta-analysis of all trials showing larger outcomes in 21/28). The combined ROR for subjective and objective outcomes was, respectively, 1.08 (95% CI, 1.04-1.13) and 1.03 (95% CI, 1.00-1.06) when comparing meta-analysis of all trials and meta-analysis of the 25% largest trials, 1.17 (95% CI, 1.11-1.22) and 1.13 (95% CI, 0.82-1.55) when comparing meta-analysis of all trials and limit meta-analysis, and 0.94 (95% CI, 0.86-1.04) and 1.03 (95% CI, 1.00-1.06) when comparing meta-analysis of all trials and meta-analysis restricted to trials at low risk of bias.

Conclusions and Relevance  Estimation of treatment outcomes in meta-analyses differs depending on the strategy used. This instability in findings can result in major alterations in the conclusions derived from the analysis and underlines the need for systematic sensitivity analyses.

Figures in this Article

Sign in

Create a free personal account to sign up for alerts, share articles, and more.

Purchase Options

• Buy this article
• Subscribe to the journal

Figures

Place holder to copy figure label and caption
Figure.
Difference in Treatment Outcomes Between Trials at High or Unclear Risk and Trials at Low Risk of Bias for Each Key Domain and Overall

A ratio of odds ratios greater than 1 indicates larger treatment outcomes for trials at high or unclear risk of bias than for trials at low risk.

Graphic Jump Location

Tables

References

Letters

CME
Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.

Multimedia

Some tools below are only available to our subscribers or users with an online account.

Sign in

Create a free personal account to sign up for alerts, share articles, and more.

Purchase Options

• Buy this article
• Subscribe to the journal

Related Content

Customize your page view by dragging & repositioning the boxes below.

See Also...
Articles Related By Topic
Related Collections
PubMed Articles
Jobs
JAMAevidence.com

Users' Guides to the Medical Literature
What are the Results of the Study?

Users' Guides to the Medical Literature
What Are the Results?

brightcove.createExperiences();