0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Review |

Comparison of Evidence of Treatment Effects in Randomized and Nonrandomized Studies

John P. A. Ioannidis, MD; Anna-Bettina Haidich, MSc; Maroudia Pappa, MSc; Nikos Pantazis, MSc; Styliani I. Kokori, MD; Maria G. Tektonidou, MD; Despina G. Contopoulos-Ioannidis, MD; Joseph Lau, MD
JAMA. 2001;286(7):821-830. doi:10.1001/jama.286.7.821.
Text Size: A A A
Published online

Context There is substantial debate about whether the results of nonrandomized studies are consistent with the results of randomized controlled trials on the same topic.

Objectives To compare results of randomized and nonrandomized studies that evaluated medical interventions and to examine characteristics that may explain discrepancies between randomized and nonrandomized studies.

Data Sources MEDLINE (1966–March 2000), the Cochrane Library (Issue 3, 2000), and major journals were searched.

Study Selection Forty-five diverse topics were identified for which both randomized trials (n = 240) and nonrandomized studies (n = 168) had been performed and had been considered in meta-analyses of binary outcomes.

Data Extraction Data on events per patient in each study arm and design and characteristics of each study considered in each meta-analysis were extracted and synthesized separately for randomized and nonrandomized studies.

Data Synthesis Very good correlation was observed between the summary odds ratios of randomized and nonrandomized studies (r = 0.75; P<.001); however, nonrandomized studies tended to show larger treatment effects (28 vs 11; P = .009). Between-study heterogeneity was frequent among randomized trials alone (23%) and very frequent among nonrandomized studies alone (41%). The summary results of the 2 types of designs differed beyond chance in 7 cases (16%). Discrepancies beyond chance were less common when only prospective studies were considered (8%). Occasional differences in sample size and timing of publication were also noted between discrepant randomized and nonrandomized studies. In 28 cases (62%), the natural logarithm of the odds ratio differed by at least 50%, and in 15 cases (33%), the odds ratio varied at least 2-fold between nonrandomized studies and randomized trials.

Conclusions Despite good correlation between randomized trials and nonrandomized studies—in particular, prospective studies—discrepancies beyond chance do occur and differences in estimated magnitude of treatment effect are very common.

Figures in this Article

Sign in

Purchase Options

• Buy this article
• Subscribe to the journal
• Rent this article ?

Figures

Figure 1. Comparison of the Summary Odds Ratio and 95% Confidence Interval in Randomized Trials vs Nonrandomized Studies for the 45 Topics
Graphic Jump Location
The topic numbers correspond to the identification numbers in Table 1. Calculations have been performed with random effects in the panel A and, for comparison, by fixed effects in the panel B. The topics have been ordered according to increasing odds ratio estimates in randomized trials using random-effects calculations. Data shown in blue indicate the topics in which the difference between randomized trials and nonrandomized studies was beyond what would be expected by chance alone. For 1 topic (No. 11), both of the summary estimates lie outside the depicted range.
Figure 2. Comparison of the Summary Odds Ratio in Randomized Trials vs Nonrandomized Studies
Graphic Jump Location
Historically controlled studies are excluded from the calculations. Calculations are performed with random effects. Odds ratios are shown in a natural logarithmic scale. Not shown is 1 topic with very large summary odds ratios (>25) for both types of designs.

Tables

References

CME
Also Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
Please click the checkbox indicating that you have read the full article in order to submit your answers.
Your answers have been saved for later.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.

Multimedia

Some tools below are only available to our subscribers or users with an online account.

3,294 Views
355 Citations

Sign in

Purchase Options

• Buy this article
• Subscribe to the journal
• Rent this article ?

Related Content

Customize your page view by dragging & repositioning the boxes below.

Articles Related By Topic
Related Collections
PubMed Articles
Jobs
JAMAevidence.com

The Rational Clinical Examination: Evidence-Based Clinical Diagnosis
Original Article: Does This Patient Have a Hemorrhagic Stroke?

The Rational Clinical Examination: Evidence-Based Clinical Diagnosis
Quick Reference

×
brightcove.createExperiences();