0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Special Communication |

Issues in Comparisons Between Meta-analyses and Large Trials

John P. A. Ioannidis, MD; Joseph C. Cappelleri, PhD, MPH; Joseph Lau, MD
JAMA. 1998;279(14):1089-1093. doi:10.1001/jama.279.14.1089.
Text Size: A A A
Published online

Context.— The extent of concordance between meta-analyses and large trials on the same topic has been investigated with different protocols. Inconsistent conclusions created confusion regarding the validity of these major tools of clinical evidence.

Objective.— To evaluate protocols comparing meta-analyses and large trials in order to understand if and why they disagree on the concordance of these 2 clinical research methods.

Design.— Systematic comparison of protocol designs, study selection, definitions of agreement, analysis methods, and reported discrepancies between large trials and meta-analyses.

Results.— More discrepancies were claimed when large trials were selected from influential journals (which may prefer trials disagreeing with prior evidence) than from already performed meta-analyses (which may target homogeneous trials) and when both primary and secondary (rather than only primary) end points were considered. Depending on how agreement was defined, kappa coefficients varied from 0.22 (low agreement) to 0.72 (excellent agreement). The correlation of treatment effects between large trials and meta-analyses varied from −0.12 to 0.76, but was more similar (0.50-0.76) when only primary end points were considered. When both the magnitude and uncertainty of treatment effects were considered, large trials disagreed with meta-analyses 10% to 23% of the time. Discrepancies were attributed to different disease risks, variable protocols, quality, and publication bias.

Conclusions.— Comparisons of large trials with meta-analyses may reach different conclusions depending on how trials and meta-analyses are selected and how end points and agreement are defined. Scrutiny of these 2 major research methods can enhance our appreciation of both for guiding medical practice.

Figures in this Article

Sign in

Create a free personal account to sign up for alerts, share articles, and more.

Purchase Options

• Buy this article
• Subscribe to the journal

First Page Preview

View Large
First page PDF preview

Figures

Graphic Jump Location
Figure 1.—Randomized controlled evidence seen in the context of trial findings and trial sample size and the effect of these factors on the selection of trials for meta-analyses (used for identifying large trials in the Villar protocol6 and the Cappelleri protocol7) and for publication in influential journals (used for identifying large trials in the LeLorier protocol8).
Graphic Jump Location
Figure 2.—Plots of the odds ratio of large trials and the odds ratio of corresponding meta-analyses in the Villar6 protocol (excluding the large trial from each meta-analysis) and the LeLorier8 protocol. Each pair of large trial and corresponding meta-analysis is shown by an ellipse with dimensions proportional to the square root of the sample size of the large study and the meta-analysis. Primary end-point comparisons are shown by shaded ellipses and secondary end points by open ellipses. Note the strong correlation in the Villar protocol and the total lack of correlation in the LeLorier protocol when all comparisons are considered, but modest correlation when only primary end points are considered. The data have been reextracted from the cited studies. Three outlier comparisons in the Villar protocol are not shown. See JAMA7 for a comparable figure for the Cappelleri protocol.

Tables

References

CME
Also Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
Please click the checkbox indicating that you have read the full article in order to submit your answers.
Your answers have been saved for later.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.

Multimedia

Some tools below are only available to our subscribers or users with an online account.

Web of Science® Times Cited: 151

Sign in

Create a free personal account to sign up for alerts, share articles, and more.

Purchase Options

• Buy this article
• Subscribe to the journal

Related Content

Customize your page view by dragging & repositioning the boxes below.

Articles Related By Topic
Related Collections
PubMed Articles
Jobs
brightcove.createExperiences();