0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Quality Issues and Standards |

Discussion Sections in Reports of Controlled Trials Published in General Medical Journals FREE

Mike Clarke, DPhil; Phil Alderson, MBChB; Iain Chalmers, DSc
[+] Author Affiliations

Author Affiliations: UK Cochrane Centre, Oxford, England.


JAMA. 2002;287(21):2799-2801. doi:10.1001/jama.287.21.2799.
Text Size: A A A
Published online

Context Reliable interpretation of the results of a controlled trial entails setting its results in the context of similar research. A previous study showed that most reports of controlled trials published in 5 general medical journals in May 1997 were deficient in this respect. We assessed the extent to which reports of controlled trials published in the same 5 journals discussed new results in light of the totality of evidence from other controlled trials.

Methods Assessment of the discussion sections in all 33 reports of randomized trials published during May 2001 in Annals of Internal Medicine, BMJ, JAMA, The Lancet, and The New England Journal of Medicine.

Results Three reports appeared to have been the first published trials to address the questions studied. In none of the remaining 30 reports were the results of the new trial discussed in the context of an updated systematic review of other trials. Although reference was made to relevant systematic reviews in 3 of these 30 reports, there was no integration, quantitative or qualitative, of the results of the new trials in an update of these reviews. In the remaining 27 reports, there was no evidence that any systematic attempt had been made to set the new results in the context of previous trials.

Conclusions Between 1997 and 2001, there was no evidence of progress in the proportion of reports of trials published in general medical journals that discussed the new results within the context of, or with reference to, up-to-date systematic reviews of relevant evidence from other controlled trials.

Anyone wishing to interpret a trial needs to know how its results compare with those of similar studies, a fact recognized by the original CONSORT (Consolidated Standards of Reporting Trials) statement, which recommended that the report of a randomized trial discuss its findings in light of the totality of relevant evidence.1 In May 1997, Annals of Internal Medicine, BMJ, JAMA, The Lancet, and The New England Journal of Medicine published 26 reports of randomized trials. Reports of apparently similar trials were found for 25 of these. In only 2 were a trial's results placed in the context of an up-to-date systematic review of relevant studies. Thus, only a small proportion of the reports provided sufficient information to permit reliable interpretation of the new results.2 We repeated our study in May 2001 to assess whether there had been any detectable improvement since then. In 1997, and again in this study, it was not our aim to assess the overall quality of the discussion sections of trial reports but simply to assess how well the results of the new trial had been placed in the context of other relevant research.

A report was eligible for inclusion as a trial if it met the following criteria. First, it was published during May 2001 as a full report or article (that is, not in the editorials, news, research letters, short reports, or correspondence sections) in Annals of Internal Medicine, BMJ, JAMA, The Lancet, or The New England Journal of Medicine. Second, on the basis of the best available information, the individuals (or other units) observed in the trial were assigned prospectively to 1 of 2 or more forms of health care by using random allocation or some quasi-random method of allocation, such as alternation, date of birth, or case record number, that is, randomized and quasi-randomized trials, as defined by the Cochrane Collaboration.3

If the discussion section of an eligible report contained an explicit attempt to identify and discuss all other similar trials, whether or not an attempt was made to combine their results quantitatively with those of the new trial, it was classified as a systematic review.

Each of us searched the relevant issues of the journals in different random order. Any reports judged to be eligible by at least 1 of us were considered for inclusion. We resolved any outstanding disagreements by discussion and included reports on which all 3 of us agreed.

Other trials that seem to have addressed the question concerned in the index report were sought by searching the Cochrane Controlled Trials Register (CENTRAL). We did not systematically search for all such trials or judge whether there was sufficient similarity between the new trial and other trials to combine them in a formal meta-analysis.

We independently assessed the discussion section of each eligible report to decide whether an attempt had been made within it to integrate the results of the new trial within a systematic review, either qualitatively or quantitatively. Such a review could have been one done previously or one done especially by the authors of the report. We resolved any disagreements in our assessments by discussion.

Thirty-three reports of randomized trials, available in the online reference list (available in PDF format), were identified in the 19 issues of these 5 journals published in May 2001. In 4 reports, the authors claimed that their study was the first to have addressed the question concerned. Reports of apparently similar trials were found for 1 of these 4.

None of the 30 reports for which there appear to be similar trials contained a discussion of the trial's results in the context of an up-to-date systematic review of earlier trials. Relevant systematic reviews were mentioned in the discussion section of 3 reports,46 but the results of the new trial had not been integrated either qualitatively or quantitatively into an update of these reviews. However, one trial6 was reported in the same issue of The Lancet as a systematic review of other relevant trials.7 In the remaining 27 reports, there was no evidence that any systematic attempt had been made to set the results of the new trials in context (Table 1).

Table Graphic Jump LocationTable. Classification of Discussion Sections in Reports of Randomized Controlled Trials Published in May 1997 and May 2001 in 5 General Medical Journals

More than 35 years ago, Austin Bradford Hill suggested that the structure of a scientific paper could usefully be conceptualized in terms of 4 questions: Why did you start? What did you do? What answer did you get? And what does it mean anyway?8 These questions are reflected in a common structure for scientific reports (sometimes called IMRaD) comprising an introduction, a description of the materials and methods, the results, and a discussion of the findings. The introduction should clarify why the study was worth starting. The discussion should indicate the contribution of the new findings to the evidence available at reporting.

The public would be served better if systematic reviews were always available before new clinical studies began, which would reduce unwanted duplication of research and help ensure that new research had been designed to build on lessons from earlier research. Another consequence would be the automatic establishment of the basis for preparing a more informative discussion section when the new study is reported.9,10

In our 2 studies, separated by 4 years, we have been unable to detect any increase in the extent to which the results of new randomized trials published in 5 prestigous general medical journals have been presented within the context of updated systematic reviews of other relevant studies (Table 1). We have not identified any other reports of empirical research assessing the extent to which this issue has been addressed. Some articles have considered other issues about the quality of discussion sections,11 but the fundamental and important issue addressed in our studies seems to have been ignored. Perhaps even more worrying, the CONSORT group appears to have weakened its initial guidance on this matter by deleting their earlier reference to the need to interpret the data from a trial "in the light of the totality of the available evidence."12

The expectation that the results of a new randomized trial will be reported in the context of an up-to-date systematic review of earlier trials does not imply that the discussion section of every report of a randomized trial should contain a full account of the material, methods, and findings of such a review. The technology already exists to enable a brief review to be included in the discussion section, with links to relevant, up-to-date, systematic reviews published elsewhere. The encouraging development by the BMJ of including a summary with each report of new research to show what is already known on a topic and what the new study adds could be extended to provide links to the evidence, such as systematic reviews, upon which these summaries are based.

Because our expectations imply radical changes in the way that research is done and reported, we expect that not all researchers, journal editors, or publishers will agree with them. However, science is cumulative, and everyone, including the public, has a right to expect that this principle will be reflected more effectively in the way that science is conducted and reported. We feel that this imposes a duty on researchers to present their results in proper context and on journal editors to require researchers to do so.

Begg C, Cho M, Eastwood S.  et al.  Improving the quality of reporting of randomized controlled trials: the CONSORT statement.  JAMA.1996;276:637-639.
Clarke M, Chalmers I. Discussion sections in reports of controlled trials published in general medical journals: islands in search of continents?  JAMA.1998;280:280-282.
Clarke M, Oxman AD. Definitions of RCTs and CCTS Cochrane Reviewers Handbook 4.1.3, appendix 5b. Oxford, England: Cochrane Library, Update Software; 2002;issue 1.
CAPRICORN Investigators.  Effect of carvedilol on outcome after myocardial infarction in patients with left-ventricular dysfunction: the CAPRICORN randomised trial.  Lancet.2001;357:1385-1390.
Eccles M, Steen N, Grimshaw J.  et al.  Effect of audit and feedback, and reminder messages on primary-care radiology referrals: a randomised trial.  Lancet.2001;357:1406-1409.
Villar J, Ba'aqeel H, Piaggio G.  et al.  WHO antenatal care randomised trial for the evaluation of a new model of routine antenatal care.  Lancet.2001;357:1551-1564.
Carroli G, Villar J, Piaggio G.  et al.  WHO systematic review of randomised controlled trials of routine antenatal care.  Lancet.2001;357:1565-1570.
Hill AB. The reasons for writing.  BMJ.1965;4:870.
Chalmers I. Improving the quality and dissemination of reviews of clinical research. In: Lock S, ed. The Future of Medical Journals: In Commemoration of 150 Years of the British Medical Journal. London, England: British Medical Journal Books; 1991:127-146.
Chalmers I, Altman DG. How can medical journals help prevent poor medical research? some opportunities presented by electronic publishing.  Lancet.1999;353:490-493.
Skelton JR, Edwards SJ. The function of the discussion section in academic medical writing.  BMJ.2000;320:1269-1270.
Moher D, Schulz KF, Altman DG, Lepage L. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials.  Lancet.2001;357:1191-1194.

Figures

Tables

Table Graphic Jump LocationTable. Classification of Discussion Sections in Reports of Randomized Controlled Trials Published in May 1997 and May 2001 in 5 General Medical Journals

References

Begg C, Cho M, Eastwood S.  et al.  Improving the quality of reporting of randomized controlled trials: the CONSORT statement.  JAMA.1996;276:637-639.
Clarke M, Chalmers I. Discussion sections in reports of controlled trials published in general medical journals: islands in search of continents?  JAMA.1998;280:280-282.
Clarke M, Oxman AD. Definitions of RCTs and CCTS Cochrane Reviewers Handbook 4.1.3, appendix 5b. Oxford, England: Cochrane Library, Update Software; 2002;issue 1.
CAPRICORN Investigators.  Effect of carvedilol on outcome after myocardial infarction in patients with left-ventricular dysfunction: the CAPRICORN randomised trial.  Lancet.2001;357:1385-1390.
Eccles M, Steen N, Grimshaw J.  et al.  Effect of audit and feedback, and reminder messages on primary-care radiology referrals: a randomised trial.  Lancet.2001;357:1406-1409.
Villar J, Ba'aqeel H, Piaggio G.  et al.  WHO antenatal care randomised trial for the evaluation of a new model of routine antenatal care.  Lancet.2001;357:1551-1564.
Carroli G, Villar J, Piaggio G.  et al.  WHO systematic review of randomised controlled trials of routine antenatal care.  Lancet.2001;357:1565-1570.
Hill AB. The reasons for writing.  BMJ.1965;4:870.
Chalmers I. Improving the quality and dissemination of reviews of clinical research. In: Lock S, ed. The Future of Medical Journals: In Commemoration of 150 Years of the British Medical Journal. London, England: British Medical Journal Books; 1991:127-146.
Chalmers I, Altman DG. How can medical journals help prevent poor medical research? some opportunities presented by electronic publishing.  Lancet.1999;353:490-493.
Skelton JR, Edwards SJ. The function of the discussion section in academic medical writing.  BMJ.2000;320:1269-1270.
Moher D, Schulz KF, Altman DG, Lepage L. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials.  Lancet.2001;357:1191-1194.

Letters

CME
Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.

Multimedia

Some tools below are only available to our subscribers or users with an online account.

Web of Science® Times Cited: 51

Related Content

Customize your page view by dragging & repositioning the boxes below.

Articles Related By Topic
PubMed Articles