0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Review |

Impact of Participant and Physician Intervention Preferences on Randomized Trials:  A Systematic Review FREE

Michael King, PhD; Irwin Nazareth, PhD; Fiona Lampe, PhD; Peter Bower, PhD; Martin Chandler, MSc; Maria Morou, MSc; Bonnie Sibbald, PhD; Rosalind Lai, MLib
[+] Author Affiliations

Author Affiliations: Department of Mental Health Sciences (Dr King, Mr Chandler, and Ms Morou), Department of Primary Care and Population Sciences (Drs Nazareth and Lampe), and the Medical Library (Ms Lai), Royal Free and University College Medical School, Royal Free Campus, London; National Primary Care Research and Development Centre, University of Manchester, Manchester (Drs Bower and Sibbald), England.

More Author Information
JAMA. 2005;293(9):1089-1099. doi:10.1001/jama.293.9.1089.
Text Size: A A A
Published online

Context Allocation on the basis of randomization rather than patient choice is the gold standard of unbiased estimates of efficacy in clinical medicine. However, randomly allocating patients to treatments that do not accord with their preferences may influence internal and external validity.

Objective To determine whether preferences affect recruitment to trials (external validity) and outcomes in trials (internal validity)

Data Sources We searched MEDLINE, EMBASE, PsycINFO, CINAHL, AMED, and the Cochrane Library for articles published between 1966 and September 2004. We also hand-searched several major medical journals, searched reference lists of relevant articles, and contacted authors of published preference designs. The 2 themes in the first filter of the search strategy were preferences and possible determinants of preferences.

Study Selection Comprehensive cohorts and 2-stage trials that measured or recorded patient or physician preference, included allocation of participants to random and preference cohorts, and followed up all participants. We excluded trials with no recording of preference; of decision aids; with measurements of preferences for economic analyses; in which patients who refused randomization were followed up without reference to preferences; and of nonclinical populations.

Data Extraction Up to 4 reviewers independently evaluated the articles, and disagreements were resolved at project steering group meetings. We extracted data on study design, measurement of preference, recruitment, attrition, and summary data on the primary outcome(s) at baseline and each follow-up point.

Data Synthesis Of 10 023 citations identified, 170 articles met screening criteria and 32 (27 comprehensive cohorts and 5 two-stage trials) were determined to be eligible and were used in the final review. Although treatment preferences led to a substantial proportion of people refusing randomization, there was less evidence of bias in the characteristics of individuals agreeing to be randomized. Differences in outcome across the trials between randomized and preference groups were generally small, particularly in large trials and after accounting for baseline measures of outcome. Therefore, there was little evidence that preferences substantially interfere with the internal validity of randomized trials.

Conclusions Preferences influence whether people participate in randomized trials, but there is little evidence that they significantly affect validity.

Figures in this Article

Randomized controlled trials (RCTs) provide the most reliable evidence for treatment efficacy.1 Although trial participants are often conceptualized as passive recipients of interventions, many have preferences for treatments under evaluation and may decline to consent to randomization. This may limit generalizability of the results to clinical populations (ie, reduce external validity). When treatments cannot be blinded, patients randomly allocated to their nonpreferred intervention may experience resentful demoralization,2 which may lead to worse outcomes, either directly (through poor adherence to treatment) or indirectly (through a negative placebo-like effect).3 Thus, preferences may introduce bias (ie, reduce internal validity). Trial designs have been developed to address such problems (Box 1), but their results have not been systematically evaluated.4,5 There is little consensus on the magnitude of preference effects or the value of information from nonrandomized “preference” cohorts. As patients become more active participants in research, this issue is crucial for continued confidence in the results of randomized trials.

Comprehensive Cohort Design4

  • Design. Patients with strong preferences are offered their treatment of choice, while those without strong preferences are randomized in the conventional fashion. All patients (whether randomized or not) are followed up in the same way.

  • External validity. Almost all eligible patients enter the study, allowing examination of patients’ characteristics with all strengths of preferences.

  • Internal validity. Preference effects (eg, randomization vs preference) are confounded although can be controlled.

  • Study administration. Potentially costly if large numbers of patients express a preference and not feasible if very few patients have a preference. A priori power calculations are difficult if there is no prestudy estimate of the percentage accepting randomization.

Two-Stage, Randomized Designs5,6

  • Design. In the Wennberg design participants are initially randomized to 2 groups: in the first they are offered a choice of treatment while in the second they are randomized to treatment. The Rücker design is similar but participants randomized to preference in the first randomization, who do not have a strong preference for a treatment, are randomized a second time to a treatment.

  • External validity. Reduced because only patients accepting randomization enter the study.

  • Internal validity. All patients are randomized, increasing internal validity. However, randomization vs preference comparisons are still subject to confounding because patients’ characteristics may determine choice of treatment.

  • Study administration. Individuals with strong preferences may refuse randomization.

We conducted a systematic review of RCTs that incorporated participants’ preferences to test 2 hypotheses: (1) preferences influence recruitment to trials and thus reduce external validity and (2) preferences influence outcomes in trials and thereby reduce internal validity.

Data Sources

We searched MEDLINE, EMBASE, PsycINFO, CINAHL, AMED, and the Cochrane Library for articles published between 1966 and September 2004. The 2 themes in the first filter of the search strategy were preferences and possible determinants of preferences. The next filter captured all types of randomized trials, meta-analyses, and systematic reviews. We also hand searched BMJ, JAMA, The Lancet, and New England Journal of Medicine; searched reference lists of relevant articles; and contacted authors of published preference designs.

Study Selection

Eligibility. We included comprehensive cohorts and 2-stage trial designs that measured or recorded patient or physician preference; allocated participants to the interventions according to their preferences and by randomization; and followed up randomized and nonrandomized cohorts. We excluded trials with no recording of preference; of decision aids; with measurements of preferences for economic analyses; in which patients who refused randomization were followed up without reference to preferences; and of nonclinical populations. We did not reject trials on the basis of conventional criteria for quality1 because these relate predominantly to concealment of randomization and were not central to our aim.

Screening Process. We identified 10 023 citations. Two authors independently screened the abstracts of each citation and identified potentially relevant citations. Disagreements were discussed at a steering group. Non–English-language articles were translated for possible inclusion. This screening process yielded 170 articles.

Assessment of Study Eligibility. Up to 4 reviewers independently evaluated the articles, and disagreements were resolved at project steering group meetings. A total of 32 articles were determined to be eligible.

Data Extraction

We extracted data on study design, preference measurement, quality of randomization, blinding, and attrition. We collected information on proportion of eligible participants recruited; numbers consenting to randomization; and agreement at each allocation in trials in which participants were allocated to randomization or preference cohorts.5,6 Primary outcomes were identified through explicit statements, study hypotheses, and reported power analyses. Otherwise, the most likely primary outcome was chosen by consensus (M.K., I.N., and F.L.). We extracted numbers of participants with primary outcome data as well as summary data on the primary outcome(s) at baseline and each follow-up. We examined the trialists’ analysis plans and the results of their analyses. When necessary, we contacted authors for further information. When this was not forthcoming, the trial was included in the review but excluded from our reanalysis. We classified the primary outcomes as subjective or objective. Subjective outcomes were defined as measures of perception of or satisfaction with treatment (directly by self-report or indirectly through clinic attendance); self-reported symptoms; or self-reported behavior. Objective outcomes were those measured independently of the participant such as results of medical tests, mortality, and clinical signs.

Notation. For interventions 1 (experimental) and 2 (control), we represent the randomization groups by R1 and R2 and the preference groups by P1 and P2. In comprehensive cohorts, participants in groups P1 and P2 are those that refused randomization while in 2-stage trials they are those that were randomized to choose their treatment.

Data Synthesis

Analysis of Preference Effects on Recruitment and Attrition. We examined whether preferences influenced recruitment to trials by examining participation rates and consent to randomization. For comprehensive cohort studies, we examined differences in baseline characteristics between patients who had preferences and those who agreed to randomization by summarizing the numbers and results of reported significance tests.

We examined attrition by ascertaining the percentage of patients lost to follow-up for the primary outcome in the randomization and preference arms of each trial. These percentages were compared between groups using a paired t test (pairing randomization and preference groups within each study) weighted for sample size.

Analysis of Preference Effects on Outcome. We examined treatment-specific differences in outcome between preference and randomization groups (R1 vs P1 and R2 vs P2) for the primary outcome by (1) summarizing the methods and results of reported analyses and (2) conducting a reanalysis using summary data. For (1) we considered all available time points for the primary outcome(s), while for (2) we prespecified a single time point and primary outcome (Table 1 and Table 2).

Table Graphic Jump LocationTable 1. Summary Description of Comprehensive Cohort Trials Included in the Review
Table Graphic Jump LocationTable 2. Summary Description of 2-Stage Trials Included in the Review

Because the trials involved a range of disorders, outcome measures, and sample sizes, we converted differences into standardized effect sizes in our reanalysis. We calculated effect sizes directly for continuous outcome variables (difference in means divided by the pooled standard deviation) and calculated log odds ratios for binary outcomes before converting them into approximate effect sizes by dividing by 1.81.49 Outcome effect sizes were categorized using absolute values as |0.20| or lower; |0.21| to |0.50|; and greater than |0.50|.50 The level of significance was set a priori at P≤.05. Data were analyzed using Excel (Microsoft, Redmond, Wash) and SPSS version 11.5 (SPSS Inc, Chicago, Ill).

We emphasize that comparisons of outcome between randomization and preference groups are subject to bias, and we were not able to adjust for confounding factors in calculation of effect sizes. Therefore, for trials with baseline measures of the outcome variable, we conducted an additional analysis in which we presented baseline and outcome effect sizes and the “net” effect size (outcome minus baseline). These trials provide the strongest evidence for preference effects on outcome.

Study Characteristics

Thirty-two trials (27 comprehensive cohort trials743 and 5 two-stage randomized designs3,4448) were identified (Table 1 and Table 2). The studies covered a range of clinical areas and interventions. The information on interventions given to participants and the method of elicitation of intervention preferences varied widely. In 5 comprehensive cohort studies, physician preference was reported as a factor in the decision to decline randomization or in the final choice of treatment (Table 1).13,2124,32,33,38,39 Parental preference was relevant in 4 studies involving children.3436,38,39

External Validity

Trial Recruitment and Acceptance of Randomization. Eleven of the 27 comprehensive cohort trials did not report the number of eligible individuals who agreed to participate (ie, in either randomization or preference groups). Among the remaining 16 studies, the participation rate ranged from 50% to 100%; 12 of 16 studies reported a participation rate higher than 80% and 9 reported higher than 95% participation. Among individuals who agreed to participate, acceptance of randomization in all 27 comprehensive cohort trials varied from 26% to 88% and was lower than 50% in 14 of 27 studies. In 2 of the 5 two-stage trials, participants were randomized (to choose the intervention or undergo a second randomization) prior to consent.3,46 Two studies provided no information44,45,47 and in one trial 62% of eligible participants agreed to be randomized at the first stage.48 Participation in the next stage (allocation to R1 or R2 and P1 or P2) ranged from 48% to 100% agreeing to choose a treatment in the preference groups and 25% to 70% accepting randomization.

Baseline Differences Between Randomization and Preference Cohorts. Twenty of the 27 comprehensive cohort designs reported at least one comparison between randomization and preference groups on baseline sociodemographic factors. At least one significant difference between randomization and preference groups was found in 9 of the 20 trials. Fifteen significant differences were reported in 89 sociodemographic comparisons, a finding greater than that expected by chance. The proportion of significant findings did not differ greatly according to sample size (7/38 and 8/51 in comparison with total sample size <300 and ≥300, respectively). Significant sociodemographic findings were as follows: compared with participants accepting randomization, those with an intervention preference were more likely to be well educated (4/9 trials tested),13,2224,28,38,39 employed (3/9 trials tested),28,34,42 white (2/9 trials tested),13,28 women (in 1 trial),42 men (in 1 trial; 12 tested for sex),20 unmarried (1/5 trials tested),2527 less likely to have siblings (in 2 trials involving children),34,38,39 and more likely to be in day care (1 trial).38,39

Twenty-two of 27 comprehensive cohort trials reported at least one comparison between randomization and preference groups on baseline clinical and health characteristics. At least one significant difference was reported by 8 of the 22 trials. Overall, 29 significant differences were found in a total of 260 clinical comparisons, a finding slightly greater than that expected by chance. The proportion of significant findings for clinical characteristics was higher among larger studies (4/92 and 25/168 in comparisons with total sample size <300 and ≥300, respectively). Of the 8 trials that reported significant differences, participants in the preference group had more severe clinical problems in 220,2527 and less severe clinical problems in 3,13,28,29 while in the remaining 3 trials there was no consistent pattern.2124,32,33

Internal Validity

Attrition. Authors provided information on attrition in both randomization and preference groups in 29 of 32 trials. Among these 29 trials, the proportion of individuals with missing data for the primary outcome was similar for randomization and preference groups. For randomization groups, the numbers of studies missing data were as follows: 0% to 9% missing data, 18 trials; 10% to 19%, 7 trials; and 20% or more, 4 trials. The corresponding numbers in the preference groups were 17, 8, and 4. The mean percentage of participants lost to follow-up was 10.5% and 8.8% for randomization and preference groups, respectively (paired t test weighted for sample size, P = .62).

Analysis of Preference and Outcome Reported By the Trialists. Nineteen of the 32 comprehensive cohort and 2-stage studies examined preference effects on outcome by performing treatment-specific or treatment-adjusted comparisons of randomization and preference groups for at least one primary outcome measure and follow-up point.3,912,14,1618,2024,2830,32,33,38,39,4248 Adjustment was made for baseline disease and/or other characteristics in only 9 of the 19 studies.3,11,12,2124,28,29,4648 Seven of the 19 studies reported at least one statistically significant difference between preference and randomization groups at outcome.3,11,12,1618,20,2224,29,44,45 In 5 of the 7 studies, the preference group was favored,3,12,1618,2124,44,45 while in the remaining 2 studies, the randomization group was favored20,29 (Box 2).

Box 2. Statistically Significant Results From Analysis of Preference and Outcome as Reported by the Trialists

Outcome comparisons in which the preference group had more favorable outcome:

  • Opting for the same procedure for participants in the medical group of a medical vs surgical termination of pregnancy trial (unadjusted for baseline factors)1618

  • Treatment acceptability and willingness to continue the same treatment in the medical group of a trial of medical vs surgical treatment of heavy menstrual bleeding (unadjusted for baseline factors)44,45

  • Program attendance outcomes in the “group program” arm of a trial of patient education in cardiovascular disease (adjusted for baseline factors)3

  • Depression score in the counseling group of a counseling vs antidepressant trial (unadjusted for baseline factors)11,12

  • Total mortality in the coronary artery bypass graft (CABG) group (and in a treatment-pooled analysis) of a CABG vs percutaneous coronary angiography trial (adjusted for baseline factors)2224

Outcome comparisons for which the randomized group had more favorable outcome:

  • Self-reported cocaine use in a trial of day hospital vs inpatient treatment of cocaine addiction (adjusted for baseline factors; treatment-pooled analysis)29

  • Roland Disability Score in the radiograph group of a trial of radiograph vs no radiograph for patients consulting general practitioners for low back pain (unadjusted for baseline factors; not consistent across follow-up times)20

Reanalysis Using Standardized Effect Sizes. We derived standardized intervention-specific preference effects (based on the difference between randomization and preference groups at outcome) using a single outcome and follow-up time for each study (Table 1 and Table 2). We were able to calculate standardized effect sizes in 25 of 32 studies. Of the 7 remaining studies, 5 provided no summary data for the preference arms,15,4042,48 while standard deviations were not given and could not be approximated in 2.20,30 Of the 25 studies for which we could calculate preference effects, 2 provided only pooled data for randomization and preference groups14,47 and 2 had only one intervention group for which a randomization vs preference comparison was applicable,9,10,46 leaving a total of 46 randomization-preference comparisons.

Preference and Outcome Ignoring Baseline DifferencesTable 3 shows the standardized effect sizes for these 46 comparisons listed by sample size and classified according to magnitude of preference effect size and type of outcome (subjective or objective). Of the 46 comparisons, only 5 (from 5 studies) were statistically significant (Table 3), all of which favored the preference arm.3,1113,19,44,45 Three of these 5 comparisons were reported as significant in the investigators’ analyses (Box 2).3,11,12,44,45 The remaining significant effects (not reported by investigators) were for change in body mass index in the group not receiving hormone therapy in a study of its use for postmenopausal women19 and cardiac mortality in the group receiving PTCA comparing it with CABG in diabetic patients with coronary disease.13 Four significant randomization vs preference comparisons from the investigators’ analyses were not observed in our reanalysis. These 4 cases are explained by omission from our reanalysis because of insufficient infomation20 ; our prespecification of a single primary outcome (multiple outcomes were tested in the investigators’ analysis)1618 ; and our analysis ignoring baseline covariates.2224,29

Table Graphic Jump LocationTable 3. Outcome Effect Size for Preference vs Randomization Cohorts by Study and Intervention Group*

We evaluated preference effects according to magnitude of effect size. The majority of effect sizes for the 46 comparisons were small: 25 (54.3%) from 18 studies were |0.20| or less, 13 (28.3%) from 10 studies were between |0.21| and |0.50|, and 8 (17.4%) from 7 studies were higher than |0.50|. Of the 8 effect sizes higher than |0.50|, 6 favored the preference and 2 the randomization groups (2 occurred in a single study but were in opposing directions for the intervention groups44,45). Furthermore, moderate or large effect sizes were much less likely to occur in large studies compared with smaller ones (effect size was >|0.20| for 33% [9/27] of comparisons based on ≥100 participants vs 63% [12/19] of comparisons based on <100 participants). There was little evidence of variation in effect size according to the objectivity of outcome measure: effect size was greater than |0.20| for 44% (7/16) of comparisons with objective outcomes compared with 47% (14/30) of comparisons with subjective outcomes. When we restricted this comparison to effect sizes based on 100 or more participants, the corresponding percentages were 25% (3/12) for objective outcomes vs 40% (6/15) for subjective outcomes.

Preference and Outcome After Accounting for Baseline Differences. In 10 of the 25 studies for which we could calculate standardized effect sizes (18 of 46 intervention-specific comparisons), we were able to evaluate preference effects after accounting for baseline differences in the primary outcome variable between randomization and preference groups. In 9 studies we could derive baseline effect sizes for the relevant outcome variable,8,11,12,1618,21,2529,36,46 while for the 10th a change from baseline measure was used as an outcome in itself.19 Only 2 of the 18 comparisons were based on objective outcome measures. Figure 1 shows baseline and outcome effect sizes for the 18 intervention-specific comparisons in order of sample size (smallest at the top of the figure). The proximity of the open and closed boxes for each comparison demonstrates the outcome effect size in relation to the baseline effect size. For most comparisons, the direction and magnitude of the baseline and outcome effect sizes were similar, suggesting no clear effect of preference on outcome over and above differences that existed at baseline. Net preference effect sizes (outcome minus baseline) are shown in Figure 2. Most net effect sizes were small: 12 (66.7%) from 9 studies were |0.20| or less, 4 (22.2%) from 3 studies were between |0.21| and |0.50|, while 2 (11.1%) from 2 studies were higher than |0.50|. The 2 large net effect sizes both favored the randomization group.28,29Figures 1 and 2 show that the studies with largest sample sizes (which give the most precise estimates) tended to have the smallest baseline, outcome, and net preference effect sizes, which were clustered around zero.

Figure 1. Baseline and Outcome Effect Sizes for Preference vs Randomization Comparisons, by Study and Intervention Group
Graphic Jump Location

Studies are ordered from smallest (top) to largest (bottom). For outcomes assessed at multiple time points, the time point highlighted in Table 1 was used. Error bars indicate 95% confidence intervals.
*Preference effects calculated from change in body mass index from baseline.

Figure 2. Net Effect Sizes for Preference vs Randomization Comparisons, by Study and Intervention Group
Graphic Jump Location

Studies are ordered from smallest (top) to largest (bottom). For outcomes assessed at multiple time points, the time point highlighted in Table 1 was used.
*Preference effects calculated from change in body mass index from baseline.

Principal Findings

Our findings indicate that participants’ preferences affect recruitment. In many trials, substantial numbers refused randomization because of preferences. There was some indication that the more “empowered” (well educated and employed) are more likely to refuse randomization because of preferences. Differences in clinical characteristics showed no consistent pattern in favor of randomization or preference groups, and sociodemographic or clinical differences were evident in only a minority of studies. Thus, although preferences reduced recruitment, it was less convincing that external validity was seriously compromised, although differences may occur in important unmeasured variables.

With regard to our second hypothesis, there was some evidence that participant or physician preferences influenced outcome in a proportion of trials. However, evidence for moderate or large preference effects was much weaker in large trials and after accounting for baseline differences. Where preference effects were evident, they were inconsistent in direction and were not clearly associated with whether the primary outcome was subjective or objective. There was no evidence that preferences influenced attrition. Therefore, although our review confirms that patients and physicians often have strong intervention preferences, it gives less support to the hypothesis that preferences significantly compromise internal validity.

Strengths and Limitations

Patient preference trials provide unique data on external and internal validity. However, there are a number of limitations to our review. Interventions and settings were diverse, which limited the extent to which we could pool data from individual studies. Twenty-seven of the 32 trials were comprehensive cohort designs and thus we can make only limited conclusions about 2-stage preference designs. Preference effects, however, might be expected to be strongest in comprehensive cohorts because they include people who refuse randomization. Seven trials did not contain sufficient data for us to calculate effect sizes. Assessment of the effect of preference on outcome is subject to confounding. The extent to which it was possible to account for this was limited. To be comprehensive, we sought studies from as far back as 1966, so studies were of variable quality. In reporting the results of investigators’ published analyses we cannot rule out the possibility that comparisons were undertaken but not published and that significant findings were more likely to be reported than nonsignificant ones. However, such a bias would result in our overestimating the importance of preference in baseline and outcome comparisons.

Consequences for Observational Studies

Our review also provides insight into the outcome of nonrandomized participants who are followed up under the same experimental conditions as are randomized participants. Thus it adds to the growing evidence that where strong preferences or ethical objections to an RCT exist, observational methods are a valuable alternative.51,52 Although RCTs remain the gold standard, the outcomes of nonrandomized cohorts best approximate to those of randomized studies when exclusion criteria are carefully defined and when prognostic factors and patients’ and professionals’ preferences are well understood. These methods could be used to investigate the effectiveness of interventions when randomization is unacceptable to clinicians and/or patients and their families.53

We conclude that intervention preferences appear to have limited impact on the external or internal validity of randomized trials.

Corresponding Author: Michael King, PhD, Royal Free and University College Medical School, Rowland Hill Street, London NW3 2PF, England (m.king@medsch.ucl.ac.uk).

Author Contributions: Dr King had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: King, Bower, Sibbald, Nazareth, Lai, Lampe.

Acquisition of data: King, Nazareth, Chandler, Lampe, Morou.

Analysis and interpretation of data: King, Bower, Sibbald, Nazareth, Chandler, Lampe, Morou.

Drafting of the manuscript: King, Bower, Nazareth, Lai, Lampe, Morou.

Critical revision of the manuscript for important intellectual content: King, Bower, Chandler, Sibbald, Nazareth, Lampe.

Statistical analysis: Nazareth, Lampe.

Obtained funding: King, Bower, Sibbald, Nazareth, Lampe.

Administrative, technical, or material support: Nazareth, Lai.

Study supervision: King, Nazareth, Lai, Lampe.

Financial Disclosures: None reported.

Funding/Support: This study was funded by a grant (98/26/03) from the National Health Service Health Technology Appraisal Programme.

Role of the Sponsor: The funding organization had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; or preparation, review, or approval of the manuscript.

Disclaimer: The views expressed are those of the authors and not necessarily those of the NHS Executive or Department of Health of the United Kingdom.

NHS Centre for Reviews and Dissemination.  Undertaking systematic reviews of research on effectiveness. In: Centre for Reviews and Dissemination’s Guidance for Those Carrying Out or Commissioning Reviews: Report 4. 2nd ed. York, England: University of York; 2001
Cook T, Campbell D. Quasi-Experimentation: Design and Analysis Issues for Field Settings. Chicago, Ill: Rand McNally; 1979
Janevic M, Janz N, Dodge J.  et al.  The role of choice in health education intervention trials: a review and case study.  Soc Sci Med. 2003;56:1581-1594
PubMed   |  Link to Article
Brewin CR, Bradley C. Patient preferences and randomised clinical trials.  BMJ. 1989;299:313-315
PubMed   |  Link to Article
Wennberg JE, Barry MJ, Fowler FJ, Mulley A. Outcomes research, PORTS, and health care reform.  Ann N Y Acad Sci. 1993;703:52-62
PubMed   |  Link to Article
Rücker G. A two stage trial design for testing treatment, self selection and treatment preference effects.  Stat Med. 1989;8:477-485
PubMed   |  Link to Article
Ashok PW, Kidd A, Flett GM, Fitzmaurice A, Graham W, Templeton A. A randomised comparison of medical abortion and surgical vacuum aspiration at 10-13 weeks gestation.  Hum Reprod. 2002;17:92-98
PubMed   |  Link to Article
Bain C, Cooper KG, Parkin ED. A partially randomised patient preference trial of microwave endometrial ablation using local anaesthesia and intravenous sedation or general anaesthesia: a pilot study.  Gynaecol Endosc. 2001;10:223-228
Link to Article
Bakker A, Spinhoven P, Balkom AJ, Vleugel L, van Dyck R. Cognitive therapy by allocation versus cognitive therapy by preference in the treatment of panic disorder  Psychother Psychosom. 2000;69:240-243
PubMed   |  Link to Article
Bakker A, Van Dyck R, Spinhoven P, Van Balkom AJ. Paroxetine, clomipramine, and cognitive therapy in the treatment of panic disorder.  J Clin Psychiatry. 1999;60:831-838
PubMed   |  Link to Article
Bedi N, Chilvers C, Churchill R.  et al.  Assessing effectiveness of treatment of depression in primary care: a partially randomised preference trial.  Br J Psychiatry. 2000;177:312-318
PubMed   |  Link to Article
Chilvers C, Dewey M, Fielding K.  et al.  Antidepressant drugs and generic counselling for treatment of major depression in primary care: randomised trial with patient preference arms.  BMJ. 2001;322:772-775
PubMed   |  Link to Article
Detre KM, Guo P, Holubkov R.  et al.  Coronary revascularisation in diabetic patients: a comparison of the randomised and observational components of the Bypass Angioplasty Revascularisation (BARI).  Circulation. 1999;99:633-640
PubMed   |  Link to Article
Gossop M, Johns A, Green L. Opiate withdrawal: inpatient versus outpatient programmes and preferred versus random assignment to treatment.  BMJ. 1986;293:103-104
PubMed   |  Link to Article
Helsing M, Bergman B, Thaning L, Hero U. Quality of life and survival in patients with advanced non-small cell lung cancer receiving supportive care plus chemotherapy with carboplatin and etoposide or supportive care only: a multicentre randomised phase III trial.  Eur J Cancer. 1998;34:1036-1044
PubMed   |  Link to Article
Henshaw RC, Naji SA, Russell IT, Templeton AA. Comparison of medical abortion with surgical vacuum aspiration: women’s preferences and acceptability of treatment.  BMJ. 1993;307:714-717
PubMed   |  Link to Article
Henshaw RC, Naji SA, Russell IT, Templeton AA. A comparison of medical abortion (using miferpristone and gemeprost) with surgical vacuum aspiration: efficacy and early medical sequelae.  Hum Reprod. 1994;9:2167-2172
PubMed
Howie FL, Henshaw RC, Naji SA, Russell IT, Templeton AA. Medical abortion or vacuum aspiration? two year follow up of a patient preference trial.  Br J Obstet Gynaecol. 1997;104:829-833
PubMed   |  Link to Article
Jensen LB, Vestergaard P, Hermann AP.  et al.  Hormone replacement therapy dissociates fat mass and bone mass and tends to reduce weight gain in early postmenopausal women: a randomized controlled 5 year clinical trial of the Danish Osteoporosis Prevention Study.  J Bone Miner Res. 2003;18:333-342
PubMed   |  Link to Article
Kendrick D, Fielding K, Bentley E, Miller P, Kerslake R, Pringle M. The role of radiography in primary care patients with low back pain of at least 6 weeks duration: a randomised (unblinded) controlled trial.  Health Technol Assess. 2001;5:1-69
PubMed
Kerry S, Dundas D, Hilton S, Rink E, Patel S, Lord J. Routine referral for radiography of patients presenting with low back pain: is patient’s outcome influenced by GP’s referral for plain radiography?  Health Technol Assess. 2000;4:i-iv,1-119
PubMed
King SB, Lembo NJ, Weintraub WS.  et al. Emory Angioplasty Versus Surgical Trial (EAST).  A randomised trial comparing coronary angioplasty with coronary bypass surgery.  N Engl J Med. 1994;331:1044-1050
PubMed   |  Link to Article
King SB, Lembo NJ, Weintraub WS, Kasinski AS, Barnhart HX, Kutner MH. Emory Angioplasty Versus Surgery trial (EAST): design, recruitment, and baseline description of patients.  Am J Cardiol. 1995;70:42C-59C
Link to Article
King SB, Barnhart HX, Kosinski HS.  et al.  Angioplasty or surgery for multivessel coronary artery disease: comparison of eligible registry and randomised patients in the EAST trial and influence of treatment selection on outcomes.  Am J Cardiol. 1997;79:1453-1459
PubMed   |  Link to Article
King M, Sibbald B, Ward E.  et al.  Randomised controlled trial of non-directive counselling, cognitive-behaviour therapy and usual general practitioner care in the management of depression as well as mixed anxiety and depression in primary care.  Health Technol Assess. 2000;1:83
PubMed
Ward E, King M, Lloyd M.  et al.  Randomised controlled trial of non-directive counselling, cognitive-behaviour therapy and usual general practitioner care for patients with depression, I: clinical effectiveness.  BMJ. 2000;321:1383-1388
PubMed   |  Link to Article
Bower P, Byford S, Sibbald B.  et al.  Randomized controlled trial of non-directive counselling, cognitive-behavioural therapy, and usual general practitioner care for patients with depression, II: cost-effectiveness.  BMJ. 2000;321:1389-1392
PubMed   |  Link to Article
McKay JR, Alterman AI, McLellan T, Snider EC, O’Brien CP. Effect of random versus non random assignment in a comparison of inpatient and day hospital rehabilitation for male alcoholics.  J Consult Clin Psychol. 1995;63:70-78
PubMed   |  Link to Article
McKay JR, Alterman AI, McLellan T, Boardman CR, Mulvaney FD, O’Brien CP. Random versus non-random assignment in the evaluation of treatment for cocaine abusers.  J Consult Clin Psychol. 1998;66:697-701
PubMed   |  Link to Article
Melchart D, Steger HG, Linde K.  et al.  Integrating patient preferences in clinical trials: a pilot study of acupuncture vs. midazolam for gastroscopy.  J Altern Complement Med. 2002;8:265-274
PubMed   |  Link to Article
Nicolaides K, Brizot ML, Patel F, Snijders R. Comparison of chorionic villus sampling and amniocentesis for fetal karyotyping at 10-13 weeks gestation  Lancet. 1994;344:435-439
PubMed   |  Link to Article
Olschewski M, Schumacher M, Davis K. Analysis of randomised and nonrandomised patients in clinical trial using the comprehensive cohort follow up study design.  Control Clin Trials. 1992;13:226-239
PubMed   |  Link to Article
CASS Principal Investigators.  Coronary Artery Surgery Study (CASS): a randomized trial of coronary artery bypass surgery: comparability of entry characteristics and survival in randomized patients and non-randomized patients meeting randomisation criteria.  J Am Coll Cardiol. 1984;3:114-128
PubMed   |  Link to Article
Paradise JL, Bluestone CD, Bachman RZ.  et al.  Efficacy of tonsillectomy for recurrent throat infection in severely affected children.  N Engl J Med. 1984;310:674-683
PubMed   |  Link to Article
Paradise JL, Bluestone CD, Rogers KD.  et al.  Efficacy of adenoidectomy for recurrent otitis media in children previously treated with tympanostomy-tube placement: results of parallel randomized and nonrandomized trials.  JAMA. 1990;263:2066-2073
PubMed   |  Link to Article
Reddihough DS, King J, Coleman G, Catanese T. Efficacy of programmes based on conductive education for young children with cerebral palsy.  Dev Med Child Neurol. 1998;40:763-770
PubMed   |  Link to Article
Riedl S, Peter B, Geiss H, Aulmann M, Bach A, Lehnert T. Mikrobiologische und klinische Wirksamkeit der selektiven Darmdekontamination bei der trandthorakalen Resektion von Oesophagus-und Kardiacarcinomen.  Chirurg. 2001;72:1160-1170
PubMed   |  Link to Article
Rovers MM, Straatman H, Ingels K, van der Wilt GJ, van den Broek P, Zielhuis GA. Generalisability of trial results based on randomised versus non-randomised allocation of OME infants to ventilation tubes or watchful waiting.  J Clin Epidemiol. 2001;54:789-794
PubMed   |  Link to Article
Rovers MM. Otitis media with effusion in infants: the effects of ventilation tubes.  Proefschrift. 2000;1:2
Schumacher M, Bastert G, Bojar H.  et al. German Breast Cancer Study Group.  Randomised “X“ trial evaluating hormonal treatment and the duration of chemotherapy in node positive breast cancer patients.  J Clin Oncol. 1994;12:2086-2093
PubMed
Schmoor C, Olschewski M, Schumacher M. Randomized and non-randomized patients in clinical trials: experience with comprehensive cohort studies.  Stat Med. 1996;15:263-271
PubMed   |  Link to Article
de C Williams AC, Nicholas MK, Richardson PH, Pither CE, Fernandes J. Generalizing from a controlled trial: the effects of patient preference versus randomization on the outcome of inpatient versus outpatient chronic pain management.  Pain. 1999;83:57-65
PubMed   |  Link to Article
Woodward J, Kelly SM. A pilot study for a randomised controlled trial of water birth versus land birth.  BJOG. 2004;111:537-545
PubMed   |  Link to Article
Cooper KG, Grant AM, Garratt AM. The impact of using a partially randomised patient preference design when evaluating alternative managements for heavy menstrual bleeding.  Br J Obstet Gynaecol. 1997;104:1367-1373
PubMed   |  Link to Article
Cooper KG, Parkin DE, Garratt AM, Grant AM. A randomised comparison of medical and hysteroscopic management in women consulting a gynaecologist for treatment of heavy menstrual loss.  Br J Obstet Gynaecol. 1997;104:1360-1366
PubMed   |  Link to Article
Kitchener HC, Burns S, Nelson L.  et al.  A randomised controlled trial of cytological surveillance versus patient choice between surveillance and colposcopy in managing mildly abnormal cervical smears.  BJOG. 2004;111:63-70
PubMed   |  Link to Article
Noel PH, Larme AC, Meyer J, Marsh G, Correa A, Pugh JA. Patient choice in diabetes education curriculum.  Diabetes Care. 1998;21:896
PubMed   |  Link to Article
Rokke PD, Tomhave JA, Jocic Z. The role of client choice and target selection in self-management therapy for depression in older adults.  Psychol Aging. 1999;14:155-169
PubMed   |  Link to Article
Chinn S. A simple method for converting an odds ratio to an effect size for use in meta-analysis.  Stat Med. 2000;19:3127-3131
PubMed   |  Link to Article
Cohen J. Statistical Power Analysis for the Behavioural Sciences2nd ed. Hillsdale, NJ: Lawrence Erlbaum Associates; 1988
Benson K, Hartz A. A comparison of observational studies and randomised controlled trials.  N Engl J Med. 2000;342:1878-1886
PubMed   |  Link to Article
Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research design.  N Engl J Med. 2000;342:1887-1892
PubMed   |  Link to Article
Pawson R, Tilley N. Realistic Evaluation. London, England: Sage; 1997

Figures

Figure 1. Baseline and Outcome Effect Sizes for Preference vs Randomization Comparisons, by Study and Intervention Group
Graphic Jump Location

Studies are ordered from smallest (top) to largest (bottom). For outcomes assessed at multiple time points, the time point highlighted in Table 1 was used. Error bars indicate 95% confidence intervals.
*Preference effects calculated from change in body mass index from baseline.

Figure 2. Net Effect Sizes for Preference vs Randomization Comparisons, by Study and Intervention Group
Graphic Jump Location

Studies are ordered from smallest (top) to largest (bottom). For outcomes assessed at multiple time points, the time point highlighted in Table 1 was used.
*Preference effects calculated from change in body mass index from baseline.

Tables

Table Graphic Jump LocationTable 1. Summary Description of Comprehensive Cohort Trials Included in the Review
Table Graphic Jump LocationTable 2. Summary Description of 2-Stage Trials Included in the Review
Table Graphic Jump LocationTable 3. Outcome Effect Size for Preference vs Randomization Cohorts by Study and Intervention Group*

References

NHS Centre for Reviews and Dissemination.  Undertaking systematic reviews of research on effectiveness. In: Centre for Reviews and Dissemination’s Guidance for Those Carrying Out or Commissioning Reviews: Report 4. 2nd ed. York, England: University of York; 2001
Cook T, Campbell D. Quasi-Experimentation: Design and Analysis Issues for Field Settings. Chicago, Ill: Rand McNally; 1979
Janevic M, Janz N, Dodge J.  et al.  The role of choice in health education intervention trials: a review and case study.  Soc Sci Med. 2003;56:1581-1594
PubMed   |  Link to Article
Brewin CR, Bradley C. Patient preferences and randomised clinical trials.  BMJ. 1989;299:313-315
PubMed   |  Link to Article
Wennberg JE, Barry MJ, Fowler FJ, Mulley A. Outcomes research, PORTS, and health care reform.  Ann N Y Acad Sci. 1993;703:52-62
PubMed   |  Link to Article
Rücker G. A two stage trial design for testing treatment, self selection and treatment preference effects.  Stat Med. 1989;8:477-485
PubMed   |  Link to Article
Ashok PW, Kidd A, Flett GM, Fitzmaurice A, Graham W, Templeton A. A randomised comparison of medical abortion and surgical vacuum aspiration at 10-13 weeks gestation.  Hum Reprod. 2002;17:92-98
PubMed   |  Link to Article
Bain C, Cooper KG, Parkin ED. A partially randomised patient preference trial of microwave endometrial ablation using local anaesthesia and intravenous sedation or general anaesthesia: a pilot study.  Gynaecol Endosc. 2001;10:223-228
Link to Article
Bakker A, Spinhoven P, Balkom AJ, Vleugel L, van Dyck R. Cognitive therapy by allocation versus cognitive therapy by preference in the treatment of panic disorder  Psychother Psychosom. 2000;69:240-243
PubMed   |  Link to Article
Bakker A, Van Dyck R, Spinhoven P, Van Balkom AJ. Paroxetine, clomipramine, and cognitive therapy in the treatment of panic disorder.  J Clin Psychiatry. 1999;60:831-838
PubMed   |  Link to Article
Bedi N, Chilvers C, Churchill R.  et al.  Assessing effectiveness of treatment of depression in primary care: a partially randomised preference trial.  Br J Psychiatry. 2000;177:312-318
PubMed   |  Link to Article
Chilvers C, Dewey M, Fielding K.  et al.  Antidepressant drugs and generic counselling for treatment of major depression in primary care: randomised trial with patient preference arms.  BMJ. 2001;322:772-775
PubMed   |  Link to Article
Detre KM, Guo P, Holubkov R.  et al.  Coronary revascularisation in diabetic patients: a comparison of the randomised and observational components of the Bypass Angioplasty Revascularisation (BARI).  Circulation. 1999;99:633-640
PubMed   |  Link to Article
Gossop M, Johns A, Green L. Opiate withdrawal: inpatient versus outpatient programmes and preferred versus random assignment to treatment.  BMJ. 1986;293:103-104
PubMed   |  Link to Article
Helsing M, Bergman B, Thaning L, Hero U. Quality of life and survival in patients with advanced non-small cell lung cancer receiving supportive care plus chemotherapy with carboplatin and etoposide or supportive care only: a multicentre randomised phase III trial.  Eur J Cancer. 1998;34:1036-1044
PubMed   |  Link to Article
Henshaw RC, Naji SA, Russell IT, Templeton AA. Comparison of medical abortion with surgical vacuum aspiration: women’s preferences and acceptability of treatment.  BMJ. 1993;307:714-717
PubMed   |  Link to Article
Henshaw RC, Naji SA, Russell IT, Templeton AA. A comparison of medical abortion (using miferpristone and gemeprost) with surgical vacuum aspiration: efficacy and early medical sequelae.  Hum Reprod. 1994;9:2167-2172
PubMed
Howie FL, Henshaw RC, Naji SA, Russell IT, Templeton AA. Medical abortion or vacuum aspiration? two year follow up of a patient preference trial.  Br J Obstet Gynaecol. 1997;104:829-833
PubMed   |  Link to Article
Jensen LB, Vestergaard P, Hermann AP.  et al.  Hormone replacement therapy dissociates fat mass and bone mass and tends to reduce weight gain in early postmenopausal women: a randomized controlled 5 year clinical trial of the Danish Osteoporosis Prevention Study.  J Bone Miner Res. 2003;18:333-342
PubMed   |  Link to Article
Kendrick D, Fielding K, Bentley E, Miller P, Kerslake R, Pringle M. The role of radiography in primary care patients with low back pain of at least 6 weeks duration: a randomised (unblinded) controlled trial.  Health Technol Assess. 2001;5:1-69
PubMed
Kerry S, Dundas D, Hilton S, Rink E, Patel S, Lord J. Routine referral for radiography of patients presenting with low back pain: is patient’s outcome influenced by GP’s referral for plain radiography?  Health Technol Assess. 2000;4:i-iv,1-119
PubMed
King SB, Lembo NJ, Weintraub WS.  et al. Emory Angioplasty Versus Surgical Trial (EAST).  A randomised trial comparing coronary angioplasty with coronary bypass surgery.  N Engl J Med. 1994;331:1044-1050
PubMed   |  Link to Article
King SB, Lembo NJ, Weintraub WS, Kasinski AS, Barnhart HX, Kutner MH. Emory Angioplasty Versus Surgery trial (EAST): design, recruitment, and baseline description of patients.  Am J Cardiol. 1995;70:42C-59C
Link to Article
King SB, Barnhart HX, Kosinski HS.  et al.  Angioplasty or surgery for multivessel coronary artery disease: comparison of eligible registry and randomised patients in the EAST trial and influence of treatment selection on outcomes.  Am J Cardiol. 1997;79:1453-1459
PubMed   |  Link to Article
King M, Sibbald B, Ward E.  et al.  Randomised controlled trial of non-directive counselling, cognitive-behaviour therapy and usual general practitioner care in the management of depression as well as mixed anxiety and depression in primary care.  Health Technol Assess. 2000;1:83
PubMed
Ward E, King M, Lloyd M.  et al.  Randomised controlled trial of non-directive counselling, cognitive-behaviour therapy and usual general practitioner care for patients with depression, I: clinical effectiveness.  BMJ. 2000;321:1383-1388
PubMed   |  Link to Article
Bower P, Byford S, Sibbald B.  et al.  Randomized controlled trial of non-directive counselling, cognitive-behavioural therapy, and usual general practitioner care for patients with depression, II: cost-effectiveness.  BMJ. 2000;321:1389-1392
PubMed   |  Link to Article
McKay JR, Alterman AI, McLellan T, Snider EC, O’Brien CP. Effect of random versus non random assignment in a comparison of inpatient and day hospital rehabilitation for male alcoholics.  J Consult Clin Psychol. 1995;63:70-78
PubMed   |  Link to Article
McKay JR, Alterman AI, McLellan T, Boardman CR, Mulvaney FD, O’Brien CP. Random versus non-random assignment in the evaluation of treatment for cocaine abusers.  J Consult Clin Psychol. 1998;66:697-701
PubMed   |  Link to Article
Melchart D, Steger HG, Linde K.  et al.  Integrating patient preferences in clinical trials: a pilot study of acupuncture vs. midazolam for gastroscopy.  J Altern Complement Med. 2002;8:265-274
PubMed   |  Link to Article
Nicolaides K, Brizot ML, Patel F, Snijders R. Comparison of chorionic villus sampling and amniocentesis for fetal karyotyping at 10-13 weeks gestation  Lancet. 1994;344:435-439
PubMed   |  Link to Article
Olschewski M, Schumacher M, Davis K. Analysis of randomised and nonrandomised patients in clinical trial using the comprehensive cohort follow up study design.  Control Clin Trials. 1992;13:226-239
PubMed   |  Link to Article
CASS Principal Investigators.  Coronary Artery Surgery Study (CASS): a randomized trial of coronary artery bypass surgery: comparability of entry characteristics and survival in randomized patients and non-randomized patients meeting randomisation criteria.  J Am Coll Cardiol. 1984;3:114-128
PubMed   |  Link to Article
Paradise JL, Bluestone CD, Bachman RZ.  et al.  Efficacy of tonsillectomy for recurrent throat infection in severely affected children.  N Engl J Med. 1984;310:674-683
PubMed   |  Link to Article
Paradise JL, Bluestone CD, Rogers KD.  et al.  Efficacy of adenoidectomy for recurrent otitis media in children previously treated with tympanostomy-tube placement: results of parallel randomized and nonrandomized trials.  JAMA. 1990;263:2066-2073
PubMed   |  Link to Article
Reddihough DS, King J, Coleman G, Catanese T. Efficacy of programmes based on conductive education for young children with cerebral palsy.  Dev Med Child Neurol. 1998;40:763-770
PubMed   |  Link to Article
Riedl S, Peter B, Geiss H, Aulmann M, Bach A, Lehnert T. Mikrobiologische und klinische Wirksamkeit der selektiven Darmdekontamination bei der trandthorakalen Resektion von Oesophagus-und Kardiacarcinomen.  Chirurg. 2001;72:1160-1170
PubMed   |  Link to Article
Rovers MM, Straatman H, Ingels K, van der Wilt GJ, van den Broek P, Zielhuis GA. Generalisability of trial results based on randomised versus non-randomised allocation of OME infants to ventilation tubes or watchful waiting.  J Clin Epidemiol. 2001;54:789-794
PubMed   |  Link to Article
Rovers MM. Otitis media with effusion in infants: the effects of ventilation tubes.  Proefschrift. 2000;1:2
Schumacher M, Bastert G, Bojar H.  et al. German Breast Cancer Study Group.  Randomised “X“ trial evaluating hormonal treatment and the duration of chemotherapy in node positive breast cancer patients.  J Clin Oncol. 1994;12:2086-2093
PubMed
Schmoor C, Olschewski M, Schumacher M. Randomized and non-randomized patients in clinical trials: experience with comprehensive cohort studies.  Stat Med. 1996;15:263-271
PubMed   |  Link to Article
de C Williams AC, Nicholas MK, Richardson PH, Pither CE, Fernandes J. Generalizing from a controlled trial: the effects of patient preference versus randomization on the outcome of inpatient versus outpatient chronic pain management.  Pain. 1999;83:57-65
PubMed   |  Link to Article
Woodward J, Kelly SM. A pilot study for a randomised controlled trial of water birth versus land birth.  BJOG. 2004;111:537-545
PubMed   |  Link to Article
Cooper KG, Grant AM, Garratt AM. The impact of using a partially randomised patient preference design when evaluating alternative managements for heavy menstrual bleeding.  Br J Obstet Gynaecol. 1997;104:1367-1373
PubMed   |  Link to Article
Cooper KG, Parkin DE, Garratt AM, Grant AM. A randomised comparison of medical and hysteroscopic management in women consulting a gynaecologist for treatment of heavy menstrual loss.  Br J Obstet Gynaecol. 1997;104:1360-1366
PubMed   |  Link to Article
Kitchener HC, Burns S, Nelson L.  et al.  A randomised controlled trial of cytological surveillance versus patient choice between surveillance and colposcopy in managing mildly abnormal cervical smears.  BJOG. 2004;111:63-70
PubMed   |  Link to Article
Noel PH, Larme AC, Meyer J, Marsh G, Correa A, Pugh JA. Patient choice in diabetes education curriculum.  Diabetes Care. 1998;21:896
PubMed   |  Link to Article
Rokke PD, Tomhave JA, Jocic Z. The role of client choice and target selection in self-management therapy for depression in older adults.  Psychol Aging. 1999;14:155-169
PubMed   |  Link to Article
Chinn S. A simple method for converting an odds ratio to an effect size for use in meta-analysis.  Stat Med. 2000;19:3127-3131
PubMed   |  Link to Article
Cohen J. Statistical Power Analysis for the Behavioural Sciences2nd ed. Hillsdale, NJ: Lawrence Erlbaum Associates; 1988
Benson K, Hartz A. A comparison of observational studies and randomised controlled trials.  N Engl J Med. 2000;342:1878-1886
PubMed   |  Link to Article
Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research design.  N Engl J Med. 2000;342:1887-1892
PubMed   |  Link to Article
Pawson R, Tilley N. Realistic Evaluation. London, England: Sage; 1997

Letters

CME
Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.

Multimedia

Some tools below are only available to our subscribers or users with an online account.

Web of Science® Times Cited: 97

Related Content

Customize your page view by dragging & repositioning the boxes below.

Articles Related By Topic
Related Collections
PubMed Articles
JAMAevidence.com