0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Peer Review Congress |

Reporting of Randomized Clinical Trial Descriptors and Use of Structured Abstracts FREE

Roberta W. Scherer, PhD; Barbara Crawley, MS
[+] Author Affiliations

From the Department of Epidemiology and Preventive Medicine, University of Maryland School of Medicine, Baltimore.


JAMA. 1998;280(3):269-272. doi:10.1001/jama.280.3.269.
Text Size: A A A
Published online

Context.— Structured abstracts, that is, abstracts that describe a study using requisite content headings, provide more informative content. Concomitant reporting in the text of the report might improve with structured abstract use because of increased awareness by authors or editors of important study areas associated with content headings.

Objective.— To assess whether structured abstract use is associated with improved reporting of randomized clinical trials.

Design and Setting.— Survey of trial reports published the year preceding, of, and following new use of structured abstracts, found by hand searching Archives of Ophthalmology (1992-1994) and Ophthalmology (1991-1993), as well as trial reports published concurrently without change in abstract format (American Journal of Ophthalmology, 1991-1994).

Main Outcome Measures.— We measured the inclusion of 56 criteria derived from Consolidated Standards of Reporting Trials (CONSORT) descriptors (JAMA 1996;276:637-639) in the text of each report and calculated the number of criteria included per report and the proportion of reports including individual criteria. Reports with structured abstracts were compared with those without, and reports published in 1993 and 1994 in the American Journal of Ophthalmology were compared with those published in 1991 and 1992.

Results.— The mean (SEM) number of criteria included by authors was 15.8 (0.4) per report in 125 trial reports. We found no difference in the mean number of criteria included or the proportion of reports that included specific criteria by journal. Following structured abstract use, there was no difference in either the mean number of criteria per report or the proportion of reports including a majority of criteria within each CONSORT subheading. Four criteria were included more often and 2 less often following structured abstract use in individual journals.

Conclusion.— Using CONSORT descriptor criteria to evaluate reporting quality, we found no difference in text reporting associated with structured abstract use in the journals examined.

Figures in this Article

STRUCTURED ABSTRACTS, that is, abstracts that describe a study using specified content headings rather that paragraph format, were suggested by the Ad Hoc Working Group for Critical Appraisal of the Medical Literature in 19871 "to provide more information . . . for articles reporting original research . . . of medical care."2 The original objectives for using structured abstracts were to help health professionals quickly assess the reliability and content of a clinical report, to facilitate peer review, and to aid accurate indexing and retrieval of reports from computerized databases such as MEDLINE and EMBASE. In structured abstracts essential elements are concisely reported using content headings (eg, objective, design, setting, participants, intervention, results, conclusions).1 Rapid adoption of structured abstracts by journals resulted in an annual doubling of reports with structured abstracts published from 1989 through 1991 appearing in MEDLINE; 15% of these were reports of clinical trials.3

Froom and Froom4 evaluated the quality of structured abstracts and found important deficiencies in reporting of patient demographics, follow-up, and statistical methods. Taddio et al5 compared structured with unstructured abstracts and found an improvement in reporting using the structured format, albeit with similar deficiencies. They could not rule out that improvements were due to changes over time. Harcourt et al3 reported that indexers at the National Library of Medicine applied an average of 3 more Medical Subject Headings to reports with structured abstracts compared with concurrent unstructured abstracts, but could not determine whether this finding was related to better abstract or text reporting or to differential indexing of the journals that first incorporated structured abstracts.

To our knowledge, no investigator has directly examined whether structured abstract use affects the reporting quality of report text. In writing about structured abstract merits, Rennie and Glass6 raise the possibility that "[abstract] structure reminds authors . . . of the necessity of providing each category of information." By focusing on the content headings required by a structured abstract, authors or editors might incorporate descriptors more consistently in the manuscript text. Our objective was to assess whether the use of structured abstracts is associated with an improvement in the overall reporting of ophthalmology randomized controlled trials (RCTs).

Materials

We selected 2 US ophthalmology journals that we previously found to regularly report large numbers of RCTs7 and that recently revised their Instructions to Authors to require structured abstracts, the Archives of Ophthalmology and Ophthalmology.8,9 We manually searched for reports describing RCTs the year preceding, of, and following the first appearance of structured abstracts (1992, 1993, and 1994, respectively, in the Archives of Ophthalmology and 1991, 1992, and 1993, respectively, in Ophthalmology). To monitor changes over time, we searched the American Journal of Ophthalmology (a US ophthalmology journal that publishes similar numbers of RCTs7 and that did not require structured abstracts until late 1994)10 for RCT reports from 1991 through 1994.

Hand searching was carried out by 2 independent, trained readers who examined each full-length report for RCT status in the selected journals for the specified years.11 We defined RCT as a controlled experiment designed to evaluate an intervention or diagnostic tool, using a random method to assign individuals, eyes, or some other unit to a test or comparison group. We included quasi-randomized clinical trials, ie, those employing a method of assignment (eg, alternation) designed to avoid bias. Reports of RCTs that did not include data by randomized treatment group (eg, validation of a method used for measuring an outcome) were excluded.

We found 154 reports of RCTs. Five papers were excluded; 1 had no abstract, 2 presented data on subsets of patients, and 2 examined methods to measure outcomes. We also excluded 24 reports because the authors stated that an abbreviated methodological description was provided since the methods had previously been described.

Extraction of RCT Design and Operational Characteristics. The Consolidated Standards of Reporting Trials (CONSORT) statement describes RCT descriptors and provides a flowchart showing patient entry and follow-up that are recommended for inclusion in every RCT report.12 Using CONSORT descriptors as a "gold standard" to evaluate reporting quality in each article, we scored the presence of 56 criteria (each flowchart block or descriptor, Table 1) in individual reports as yes, no, or not applicable. We selected 9 criteria (with daggers in Table 1) corresponding to common abstract content headings to measure inclusion in each abstract.

Table Graphic Jump LocationTable 1.—CONSORT Descriptor Criteria*

Because some study- or journal-specific characteristics could influence the inclusion of CONSORT statement descriptors, we extracted information by journal about study and report characteristics, including purpose of intervention, multicenter status, type of test intervention, sample size, group or individual authorship, length of report (number of pages), and length of methods section (number of pages).

Analyses. Data were entered into a database (Paradox, Version 4.0, Borland International Inc, Scotts Valley, Calif), and exported to a statistical program (SAS, Version 6.2, SAS Institute Inc, Cary, NC). We calculated the number of criteria included in each report, the proportion of reports that included specific criteria, and the number of reports that included more than the majority of criteria within each CONSORT subheading (introduction, protocol, etc). We compared structured with unstructured abstracts, and reports published in 1991 and 1992 with those published in 1993 and 1994 (American Journal of Ophthalmology), using the Student t test or χ2 tests. Odds ratios (ORs) with 95% confidence intervals (CIs) are presented.

Description of RCTs by Journal

We found that RCTs reported in the Archives of Ophthalmology, compared with Ophthalmology and the American Journal of Ophthalmology, were more often multicentered (10 [38%] of 26 vs 12 [24%] of 51 and 13 [27%] of 48, respectively; P=.07), evaluated surgical or laser trials more often (7 [27%] of 26 vs 6 [12%] of 51 and 3 [6%] of 48, respectively; P=.001), more frequently devoted more than a single page to the methods section (10 [38%] of 26 vs 9 [18%] of 51 and 12 [25%] of 48, respectively; P=.005), and published more reports with study group authorship (5 [19%] of 26 vs 2 [4%] of 51 and 3 [6%] of 48, respectively; P=.001). Thus, we examined all results separately by journal.

Reporting of Descriptors in Text by Journal. Reporting of CONSORT criteria in the text was unimpressive. The mean (SEM) number of criteria included was 15.8 (0.4) of a possible 56; there was little difference among journals (Table 1). Journals were also remarkably similar in the proportion of reports that included specific criteria (Figure 1). Criteria reported in a low proportion of reports in all 3 journals were often associated with CONSORT subheadings associated with RCT methods, such as assignment and masking.

Graphic Jump Location
Inclusion of Consolidated Standards of Reporting Trials (CONSORT) descriptors by journal. Data shown are percentages of all reports for all years from a single journal including individual criteria (numbered as in Table 1), grouped by category, and calculated as yes÷(yes+no) responses.

Comparison by Structured Abstract Use or Over Time. We found no difference in the mean number of criteria that were included in reports with structured abstracts compared with those without (Table 2). Including a majority of criteria within a single CONSORT subheading was positively associated with structured abstracts for "protocol" in the Archives of Ophthalmology (OR, 2.16; 95% CI, 1.12-4.16), negatively associated for "introduction" in Ophthalmology (OR, 0.57; 95% CI, 0.33-0.95), and not associated with later year of publication. Thus, we found no evidence for improvement in inclusion of criteria associated with structured abstracts.

Table Graphic Jump LocationTable 2.—Criteria Included in Text by Journal and Structured Abstract Use or Year of Publication

We then calculated the proportion of reports in which a specific criterion was included to see if there were improved reporting by criterion associated with structured abstract use or year of publication. Individual criteria were included more or less often following structured abstract use or with later publication years, but by individual journal (Table 3).

Table Graphic Jump LocationTable 3.—Criteria Included in a Significantly Larger or Smaller Proportion of Reports Associated With Structured Abstracts or Later Year of Publication

Reporting of Descriptors in Abstract of Report. Of the 9 criteria used to evaluate abstract reporting, a mean of 5.0 (0.2) were included in all abstracts from all journals. Structured abstracts were more often associated with inclusion of criteria in the Archives of Ophthalmology (Table 2).

Specific criteria included infrequently in structured abstracts for the Archives of Ophthalmology, Ophthalmology, and American Journal of Ophthalmology were description study population (6 [67%] of 9, 18 [57%] of 28, and 16 [64%] of 25, respectively); primary outcome (6 [67%] of 9, 8 [29%] of 28, and 12 [48%] of 25, respectively); and number of patients followed up (2 [22%] of 9, 5 [18%] of 28, and 3 [12%] of 25, respectively).

Our results do not support an association of improved text reporting with structured abstracts or later publication in the journals examined. These 3 journals tended to be more alike than different in overall reporting of individual criteria, and there was no consistent pattern of change in reporting associated with structured abstract use or later publication. Possibly, the time period we examined was transitional or represented a lag time while editors or authors were incorporating use of structured abstracts. Haynes et al2 reported that initial use of structured abstracts found some authors writing abstracts concurrently with manuscript preparation, whereas others did so only at submission or on editor request. Also, no special emphasis was placed on use of structured abstracts for RCTs initially, even with recognized importance for trial reports. Finally, it is possible that our sample size was insufficient to detect subtle changes in reporting, as our results are based on 125 reports from 3 ophthalmology journals; they may not be generalizable to other journals or areas of medicine.

Although we did not detect improvement in overall text reporting, we thought there might be individual criteria reported more frequently when structured abstracts were used, but found no consistent reporting pattern. For example, use of random or trial in the title and rationale for statistical tests were reported more often, but in separate journals. Some criteria were reported less frequently with structured abstracts, and perhaps were viewed as less important when space constraints limited text length. However, any changes we report in inclusion of criteria associated with structured abstract use or later year of publication may be due to chance, given the number of observations.

Although checklists for assessing reporting quality of RCTs were available,1315 we chose to use CONSORT descriptors as a "gold standard" since it comprises a comprehensive list of criteria. We did not intend to evaluate trial quality. It has been argued, however, that "a well-designed but poorly reported trial could be judged as having low quality,"16 so assessing reporting is an important first step in assessing trial quality.

Our initial search yielded 24 RCT reports that included abbreviated methodological descriptions because methods had previously been reported. Editors are faced with a tension between space limitations and inclusion of all CONSORT descriptors in subsequent RCT reports. Since readers may not have previous reports available, we believe each RCT report should include all CONSORT descriptors to allow independent report evaluation.

Finally, we found a significant improvement in abstract reporting quality when structured abstracts were used in the Archives of Ophthalmology, and some improvement in Ophthalmology. Consistent with findings of others,4,5 we found abstract reporting deficiencies with authors frequently omitting a description of study population, primary outcome, or number of patients followed up.

In summary, we found no improvement in text reporting when structured abstracts were used. Nevertheless, structured abstract use should not be abandoned since abstract reporting itself is improved using this format. Rather, our results highlight the need for a standard such as the CONSORT statement to enhance RCT text reporting.

Ad Hoc Working Group for Critical Appraisal of the Medical Literature.  A proposal for more informative abstracts of clinical articles.  Ann Intern Med.1987;106:595-604.
Haynes RB, Mulrow CD, Huth EJ, Altman DG, Gardner MJ. More informative abstracts revisited.  Ann Intern Med.1990;113:69-76.
Harcourt AM, Knecht LS, Humphreys BL. Structured abstracts in MEDLINE, 1989-1991.  Bull Med Libr Assoc.1995;83:190-195.
Froom P, Froom J. Deficiencies in structured medical abstracts.  J Clin Epidemiol.1993;46:591-594.
Taddio A, Pain T, Fassos FF, Boon H, Ilersich AL, Elnarson TR. Quality of nonstructured and structured abstracts of original research articles in the British Medical Journal, the Canadian Medical Association Journal and the Journal of the American Medical Association Can Med Assoc J.1994;150:1611-1615.
Rennie D, Glass RM. Structuring abstracts to make them more informative.  JAMA.1991;266:116-117.
Dickersin K, Scherer RW, Lefebvre C. Identification of relevant studies for systematic reviews.  BMJ.1994;309:1286-1291.
Goldberg MF. Changes in the Archives Arch Ophthalmol.1993;111:39-40.
Lichter PR. Structured abstract now required for all submissions to the journal.  Ophthalmology.1991;98:1611-1612.
Straastsma BR. Information for authors and benefits to readers.  Am J Ophthalmol.1994;117:104-105.
 Cochrane Hand Search Manual . Available at: http://hiru.mcmaster.ca/cochrane/registry/handsrch/hsmanual.htm. Updated March 25, 1996.
Begg C, Eastwood S, Horton R.  et al.  Improving the quality of reporting of randomized controlled trials.  JAMA.1996;276:637-639.
Mahon WA, Daniel EE. A method for the assessment of reports of drug trials.  Can Med Assoc J.1964;90:565-569.
DerSimonian R, Charette LJ, McPeek B, Mosteller F. Reporting on methods in clinical trials.  N Engl J Med.1982;306:1332-1337.
Grant A. Reporting clinical trials.  Br J Obstet Gynaecol.1989;96:397-400.
Moher D, Jadad AR, Tugwell P. Assessing the quality of randomized controlled trials: current issues and future directions.  Int J Technol Assess Health Care.1996;12:195-208.

Figures

Graphic Jump Location
Inclusion of Consolidated Standards of Reporting Trials (CONSORT) descriptors by journal. Data shown are percentages of all reports for all years from a single journal including individual criteria (numbered as in Table 1), grouped by category, and calculated as yes÷(yes+no) responses.

Tables

Table Graphic Jump LocationTable 1.—CONSORT Descriptor Criteria*
Table Graphic Jump LocationTable 2.—Criteria Included in Text by Journal and Structured Abstract Use or Year of Publication
Table Graphic Jump LocationTable 3.—Criteria Included in a Significantly Larger or Smaller Proportion of Reports Associated With Structured Abstracts or Later Year of Publication

References

Ad Hoc Working Group for Critical Appraisal of the Medical Literature.  A proposal for more informative abstracts of clinical articles.  Ann Intern Med.1987;106:595-604.
Haynes RB, Mulrow CD, Huth EJ, Altman DG, Gardner MJ. More informative abstracts revisited.  Ann Intern Med.1990;113:69-76.
Harcourt AM, Knecht LS, Humphreys BL. Structured abstracts in MEDLINE, 1989-1991.  Bull Med Libr Assoc.1995;83:190-195.
Froom P, Froom J. Deficiencies in structured medical abstracts.  J Clin Epidemiol.1993;46:591-594.
Taddio A, Pain T, Fassos FF, Boon H, Ilersich AL, Elnarson TR. Quality of nonstructured and structured abstracts of original research articles in the British Medical Journal, the Canadian Medical Association Journal and the Journal of the American Medical Association Can Med Assoc J.1994;150:1611-1615.
Rennie D, Glass RM. Structuring abstracts to make them more informative.  JAMA.1991;266:116-117.
Dickersin K, Scherer RW, Lefebvre C. Identification of relevant studies for systematic reviews.  BMJ.1994;309:1286-1291.
Goldberg MF. Changes in the Archives Arch Ophthalmol.1993;111:39-40.
Lichter PR. Structured abstract now required for all submissions to the journal.  Ophthalmology.1991;98:1611-1612.
Straastsma BR. Information for authors and benefits to readers.  Am J Ophthalmol.1994;117:104-105.
 Cochrane Hand Search Manual . Available at: http://hiru.mcmaster.ca/cochrane/registry/handsrch/hsmanual.htm. Updated March 25, 1996.
Begg C, Eastwood S, Horton R.  et al.  Improving the quality of reporting of randomized controlled trials.  JAMA.1996;276:637-639.
Mahon WA, Daniel EE. A method for the assessment of reports of drug trials.  Can Med Assoc J.1964;90:565-569.
DerSimonian R, Charette LJ, McPeek B, Mosteller F. Reporting on methods in clinical trials.  N Engl J Med.1982;306:1332-1337.
Grant A. Reporting clinical trials.  Br J Obstet Gynaecol.1989;96:397-400.
Moher D, Jadad AR, Tugwell P. Assessing the quality of randomized controlled trials: current issues and future directions.  Int J Technol Assess Health Care.1996;12:195-208.
CME
Also Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
Please click the checkbox indicating that you have read the full article in order to submit your answers.
Your answers have been saved for later.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.

Multimedia

Some tools below are only available to our subscribers or users with an online account.

Web of Science® Times Cited: 43

Related Content

Customize your page view by dragging & repositioning the boxes below.