0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Brief Report |

Value of Flow Diagrams in Reports of Randomized Controlled Trials FREE

Matthias Egger, MD; Peter Jüni, MD; Christopher Bartlett, PhD; for the CONSORT Group
[+] Author Affiliations

Author Affiliations: Medical Research Council Health Services Research Collaboration, Department of Social Medicine, University of Bristol, Bristol, England (Drs Egger, Jüni, and Bartlett); and Department of Social and Preventive Medicine, University of Berne, Berne, Switzerland (Drs Egger and Jüni).


JAMA. 2001;285(15):1996-1999. doi:10.1001/jama.285.15.1996.
Text Size: A A A
Published online

Context Diagrams of the flow of participants through a clinical trial are recommended in the Consolidated Standards for Reporting of Trials (CONSORT) statement, but it is unclear whether such flow diagrams improve the quality of trial reports.

Objective To examine the information contributed by flow diagrams and the completeness of reporting overall in reports of randomized controlled trials (RCTs) published in 5 general and internal medicine journals.

Design and Setting Analysis of 270 reports of RCTs published in 1998 in the Annals of Internal Medicine (AIM; n = 19), BMJ (n = 42), JAMA (n = 45), The Lancet (n = 81), and The New England Journal of Medicine (NEJM; n = 83).

Main Outcome Measures Proportion of reports that included a flow diagram, information provided in flow diagrams, and completeness of reporting about flow of participants overall in flow diagrams or text.

Results A total of 139 reports (51.5%) of RCTs included a flow diagram, but this varied widely among journals (AIM, 21.0%; BMJ, 38.1%; JAMA, 80.0%; The Lancet, 93.8%; and NEJM, 8.4%). Diagrams generally provided useful information, but only 73 (52.5%) included the number of participants who received allocated interventions and only 32 (23.0%) included the number of participants included in the analysis. In logistic regression analysis, overall completeness of reporting about flow of study participants was associated with publication of a flow diagram.

Conclusions Flow diagrams are associated with improved quality of reporting of randomized controlled trials. However, the structure of current flow diagrams is less than ideal. We propose a revised flow diagram that includes all important counts through the stages of parallel group trials.

Figures in this Article

The Consolidated Standards for Reporting of Trials (CONSORT) criteria were published in 19961 to assist authors in reporting randomized controlled trials (RCTs). The CONSORT statement consists of a checklist for items in the methods, results, and discussion sections of a trial report and a diagram to show the flow of participants through the various stages of the trial (Figure 1, A). The CONSORT statement has been endorsed by an increasing number of journals,25 but it is unclear whether the use of CONSORT has improved the quality of reports of RCTs. We examined to what extent 5 leading medical journals adopted flow diagrams, analyzed the information contributed by these diagrams, and assessed completeness of reporting overall. The results guided our attempt to improve the design of the flow diagram (Figure 1, B).

Figure. Original and Proposed Revised Versions of CONSORT Flow Chart
Graphic Jump Location

One of us (C.B.) searched each issue published in 1998 of the Annals of Internal Medicine (AIM), BMJ, JAMA, The Lancet, and The New England Journal of Medicine (NEJM) for published reports of RCTs. A study was defined as an RCT if the assignment of participants to interventions was described as randomized by words such as randomly, random, and randomization. Two of us (M.E., P.J.) independently extracted data on the characteristics of reports and examined whether a flow diagram was included. Articles were examined in random sequence with 1 assessor examining trials in the opposite order. We searched each journal's Web site for diagrams that were published electronically but did not appear in print.

We examined whether 6 counts were provided in diagrams: (1) number of patients assessed for eligibility, (2) number found to be eligible, (3) number randomized (per group), (4) number who received allocated intervention (per group), (5) number who were lost to follow-up (per group), and (6) number included in the main analysis of primary outcomes (per group). In a separate step, we assessed whether these counts were provided either in a flow diagram or anywhere else in the article.

Each article was read independently by 2 of us in an incomplete randomized Latin square design, and interrater reliability6 for assessment of information in flow diagrams and overall was determined. Disagreements were resolved by consensus. We compared overall completeness of reporting between journals using χ2 tests and used logistic regression to assess the association of flow diagrams with completeness of reporting.

The hand search of these journals identified 290 articles. We excluded 1 quasi-randomized trial, 10 cluster trials, and 9 reports for which the focus was not on randomized comparisons. Our study sample thus consisted of 270 reports of RCTs. Most trials were of parallel group design (256 [94.8%]) and evaluated pharmacological interventions (173 [64.1%]). A total of 139 reports (51.5%) included a flow diagram but this varied widely among journals (Table 1). Five (29.4%) of 17 short reports included a flow diagram. No additional diagrams were found on the journals' Web sites. Interrater reliability6 for the assessment of information provided in flow diagrams was good (κ values ranging from 0.60 to 0.81), but showed more disagreement for assessment of overall completeness of reporting (κ, 0.21-0.71).

Table Graphic Jump LocationTable 1. Information on Participant Flow Provided in 270 Reports of Randomized Controlled Trials Published in 5 Journals in 1998*

Characteristics of flow diagrams were similar across journals. Sixty-five diagrams (46.8%) included the number of patients assessed for eligibility, and 94 (67.6%) reported the number found to be eligible. The number assigned to each study group was reported in 129 flow diagrams (92.8%), and 73 (52.5%) indicated whether interventions had been received as allocated. Most flow diagrams (114 [82.0%]) indicated how many patients in each group were lost to follow-up, but only 32 (23.0%) reported the number included in the analysis.

Overall reporting was more complete in journals that publish flow diagrams more frequently (Table 1). However, there is clearly room for improvement in all journals, in particular with regard to the number receiving the allocated intervention and the number lost to follow-up. Independent of journal, reporting tended to be less complete if the trial was published as a short report. In logistic regression analysis, completeness of reporting was associated with the publication of a flow diagram (Table 2). In multivariable analyses, there was little evidence for confounding by journal, and differences among journals were largely explained by the publication or nonpublication of flow diagrams.

Table Graphic Jump LocationTable 2. Association of Presence of a Flow Diagram With Completeness of Reporting*

A diagram showing the flow of participants from enrollment to analysis is an important element of the CONSORT standards for the reporting of clinical trials. Flow diagrams provide an aid to trialists when writing trial results and assist readers in the critical appraisal of the internal and external validity of a trial. We found that the information on the progress of participants through the trial was more complete in journals that frequently publish flow diagrams and that the publication of these diagrams is associated with better reporting in individual articles. Our results are consistent with those of Moher and colleagues,7 who compared reports published in the same journals before and after the CONSORT statement became available. The proportion of articles that included a flow diagram varied widely across journals and was lower in short reports. Flow diagrams take up precious journal space, and editors may sometimes feel that this space is better used otherwise. Our findings provide strong support for the idea that RCTs always should be published as full articles8 including a flow diagram.

The shortcomings of the flow diagram template recommended in 19961 may be another reason some editors are reluctant to publish flow diagrams. Meinert9 pointed out that the terms used in the 1996 CONSORT statement lacked clarity and that the information presented in the flow diagram was incomplete. Our results indicate that there were problems with both the original design of the flow diagram and its implementation by authors. For example, most flow diagrams provided the number of individuals randomized, although this count was not explicitly requested.9 Conversely, only about half of flow diagrams included the number of participants who actually received treatments as allocated, an item included in the original template.1 The number of participants included in the main analysis was not an item in the recommended flow diagram, and this number was included in only a few diagrams (23.0%). This finding is of concern because the latter count is essential for appraising whether a trial has been analyzed by intention to treat. A recent study found that intention-to-treat approaches are often inadequately described and inadequately applied.10 The number of persons assessed for eligibility was also frequently missing. Although this number is not relevant for assessing the internal validity of a trial, it is useful to estimate whether trial participants were likely to be representative of all patients seen.11

The CONSORT criteria are an evolving tool designed to help improve the quality of reporting of RCTs.12 Based on the findings of our present study and earlier criticism of CONSORT,9 we submit that the flow diagram is useful but in need of revision. We revised the template in collaboration with the CONSORT Group (Figure 1).13 Three tiers of boxes cover the enrollment phase and 1 box each is set apart for allocation of interventions, follow-up, and analysis. At enrollment a distinction is made between the exclusion of persons not meeting the criteria specified in the protocol and persons excluded for other reasons. The revised flow diagram explicitly includes the number of participants allocated to each group, the number initially receiving the intervention as assigned, and, if applicable, the reasons why some participants did not receive allocated interventions. Information on follow-up includes the number of participants lost to follow-up and the number of patients who stopped interventions along with the reasons why this occurred. The bottom tier of boxes includes the number of patients included in the main analysis and, if applicable, the reasons why some patients were excluded. We trust that this template could be improved further and would appreciate comments from readers through the CONSORT Web site.14

Begg C, Cho M, Eastwood S.  et al.  Improving the quality of reporting of randomized controlled trials: the CONSORT statement.  JAMA.1996;276:637-639.
Altman DG. Better reporting of randomised controlled trials: the CONSORT statement.  BMJ.1996;313:570-571.
 A standard method for reporting randomized medical scientific research: the "consolidation of the standards of reporting trials" (CONSORT) [in Dutch].  Ned Tijdschr Geneeskd.1998;142:1089-1091.
Huston P, Hoey J. CMAJ endorses the CONSORT statement: Consolidation of Standards for Reporting Trials.  CMAJ.1996;155:1277-1282.
Ausejo M, Saenz A, Moher D. CONSORT: an attempt to improve the quality of publication of clinical trials [in Spanish].  Aten Primaria.1998;21:351-352.
Landis JR, Koch GG. The measurement of observer agreement for categorical data.  Biometrics.1977;33:159-174.
Moher D, Jones A, LePage L.for the CONSORT Group.  Use of the CONSORT statement and quality of reports of randomized trials: a comparative before-and-after evaluation.  JAMA.2001;285:1992-1995.
Deeks JJ, Altman DG. Inadequate reporting of controlled trials as short reports [letter].  Lancet.1998;352:1908.
Meinert CL. Beyond CONSORT: need for improved reporting standards for clinical trials.  JAMA.1998;279:1487-1489.
Hollis S, Campbell F. What is meant by intention to treat analysis? survey of published randomised controlled trials.  BMJ.1999;319:670-674.
Jüni P, Altman DG, Egger M. Assessing the quality of controlled clinical trials. In: Egger M, Smith DG, Altman DG, eds. Systematic Reviews in Health Care: Meta-analysis in Context. London, England: BMJ Publishing Group; 2001.
Moher D. CONSORT: an evolving tool to help improve the quality of reports of randomized controlled trials.  JAMA.1998;279:1489-1491.
Moher D, Schulz KF, Altman D. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials.  JAMA.2001;285:1987-1991.
 CONSORT Web site. Available at: http://consort-statement.org. Accessed September 6, 2000.

Figures

Figure. Original and Proposed Revised Versions of CONSORT Flow Chart
Graphic Jump Location

Tables

Table Graphic Jump LocationTable 1. Information on Participant Flow Provided in 270 Reports of Randomized Controlled Trials Published in 5 Journals in 1998*
Table Graphic Jump LocationTable 2. Association of Presence of a Flow Diagram With Completeness of Reporting*

References

Begg C, Cho M, Eastwood S.  et al.  Improving the quality of reporting of randomized controlled trials: the CONSORT statement.  JAMA.1996;276:637-639.
Altman DG. Better reporting of randomised controlled trials: the CONSORT statement.  BMJ.1996;313:570-571.
 A standard method for reporting randomized medical scientific research: the "consolidation of the standards of reporting trials" (CONSORT) [in Dutch].  Ned Tijdschr Geneeskd.1998;142:1089-1091.
Huston P, Hoey J. CMAJ endorses the CONSORT statement: Consolidation of Standards for Reporting Trials.  CMAJ.1996;155:1277-1282.
Ausejo M, Saenz A, Moher D. CONSORT: an attempt to improve the quality of publication of clinical trials [in Spanish].  Aten Primaria.1998;21:351-352.
Landis JR, Koch GG. The measurement of observer agreement for categorical data.  Biometrics.1977;33:159-174.
Moher D, Jones A, LePage L.for the CONSORT Group.  Use of the CONSORT statement and quality of reports of randomized trials: a comparative before-and-after evaluation.  JAMA.2001;285:1992-1995.
Deeks JJ, Altman DG. Inadequate reporting of controlled trials as short reports [letter].  Lancet.1998;352:1908.
Meinert CL. Beyond CONSORT: need for improved reporting standards for clinical trials.  JAMA.1998;279:1487-1489.
Hollis S, Campbell F. What is meant by intention to treat analysis? survey of published randomised controlled trials.  BMJ.1999;319:670-674.
Jüni P, Altman DG, Egger M. Assessing the quality of controlled clinical trials. In: Egger M, Smith DG, Altman DG, eds. Systematic Reviews in Health Care: Meta-analysis in Context. London, England: BMJ Publishing Group; 2001.
Moher D. CONSORT: an evolving tool to help improve the quality of reports of randomized controlled trials.  JAMA.1998;279:1489-1491.
Moher D, Schulz KF, Altman D. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials.  JAMA.2001;285:1987-1991.
 CONSORT Web site. Available at: http://consort-statement.org. Accessed September 6, 2000.
CME
Also Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
Please click the checkbox indicating that you have read the full article in order to submit your answers.
Your answers have been saved for later.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.

Multimedia

Some tools below are only available to our subscribers or users with an online account.

Web of Science® Times Cited: 189

Related Content

Customize your page view by dragging & repositioning the boxes below.

See Also...
Articles Related By Topic
Related Collections
PubMed Articles