0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Publication Bias |

Association Between Time Interval to Publication and Statistical Significance FREE

Kay Dickersin, MA, PhD; Carin M. Olson, MD; Drummond Rennie, MD; Deborah Cook, MD, MSc, FRCPC; Annette Flanagin, RN, MA; Qi Zhu, MS; Jennifer Reiling; Brian Pace, MA
[+] Author Affiliations

Author Affiliations: Department of Community Health, Brown University, Providence, RI (Dr Dickersin and Ms Zhu); JAMA, Chicago, Ill (Drs Olson, Rennie, and Cook, Mss Flanagin and Reiling, and Mr Pace); Department of Medicine, Division of Emergency Medicine, University of Washington, Seattle (Dr Olson); Institute for Health Policy Studies, University of California-San Francisco (Dr Rennie); Departments of Medicine, Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario (Dr Cook).


JAMA. 2002;287(21):2829-2831. doi:10.1001/jama.287.21.2829.
Text Size: A A A
Published online

ABSTRACT

Context Studies have shown that reports from clinical trials with statistically significant results tend to be submitted and published more rapidly than reports from studies with null or nonsignficant findings.

Methods We wanted to determine whether manuscripts reporting positive results from controlled clinical trials are published more quickly than those reporting negative results. We tracked manuscripts from submission to JAMA until the publication decision. We classified results as positive if a statistically significant difference was reported for the primary outcome. Manuscripts were further classified according to indicators of methodologic quality and other study characteristics. We included manuscripts if they were submitted from February 1996 through August 1999, reported results of a prospective study in which participants were assigned to a treatment or comparison group, used statistical tests to compare differences between groups, and were accepted for publication.

Results One hundred thirty-three manuscripts met our inclusion criteria, 78 (59%) reported positive results, 51 (38%) reported negative results. We were unable to classify the direction of results for 4 articles (3%). The time interval between submission and publication was not associated with positive results: median time between submission and publication was 7.8 months for reports with positive vs 7.6 months for reports with negative results (P = .44). Time to publication also was not associated with any marker of study quality or study characteristic.

Conclusions Among 133 published controlled trials, time to publication was not associated with statistical significance, methodologic quality, or other study characteristics.

Figures in this Article

Publication bias is the tendency to publish research results based on the strength and direction of a study's findings. Data from a number of studies1 have clearly shown that publication bias exists. At least 3 studies have indicated that studies with positive results may be published more quickly than those with null or negative findings.24 However, these studies did not examine whether this increase in publication time resulted from investigators' taking less time to submit manuscripts that report positive results or resulted from the editorial review's and revision process's taking less time, or both.

METHODS

We hypothesized that controlled clinical trial articles reporting positive results are published more rapidly than those reporting negative results. The details of our methods are reported in detail elsewhere.5

Briefly, we tracked a cohort of manuscripts from submission to JAMA to acceptance. Articles were included in our study if they were submitted from February 1996 through August 1999, reported results of a prospective study in which participants were assigned to a treatment or comparison group, reported use of statistical tests to compare differences between groups, and were accepted for publication.

Outcomes were classified as positive when P<.05, the 95% confidence interval [CI] for a risk difference excluded 0, or when an odds ratio or relative risk excluded 1.0 for the primary outcome (or the majority of outcomes if a primary outcome could not be identified). Otherwise, manuscripts were classified as having negative results. Manuscripts were further classified according to indicators of methodologic quality, including reporting a sample size calculation, the method by which subjects were assigned to treatment, whether participants were masked to group assignment, whether withdrawals were reported, and whether results were analyzed by treatment received or treatment assigned. Other study characteristics included number of study centers recruiting patients, countries in which participants were enrolled, funding source, sample size, and the sex of the first author. Data were abstracted from each manuscript independently by 2 of 3 investigators (C.M.O., D.R., or D.C.) and disagreements were resolved by telephone. Additional information on publication was extracted from JAMA's database.

To examine the association between study findings and time to publication, we performed Kaplan Meier analysis, which included all published manuscripts with findings that could be classified as positive or negative. We also performed Kaplan Meier analyses examining the association between each of the study and methodologic quality characteristics for association with time interval to publication.

The University of Washington's human subjects review committee approved the study.

RESULTS

One hundred thirty-three manuscripts met our inclusion criteria, 78 (59%) reported positive results and 51 (38%) reported negative results. We were unable to classify the direction of results for 4 articles (3%). After omitting studies with unclear results, we found no evidence that articles with positive findings were published at a faster pace than those reporting negative findings: the median time interval between submission and publication was 7.8 months for positive vs 7.6 months for negative studies; (P = .44) (Figure 1).

Figure. Time Interval Between Submission and Publication by Results
Graphic Jump Location
Kaplan Meier survival analysis P = .44.

We also found no association between each of the methodologic quality and study characteristics and the time interval between submission and publication (data not shown).

COMMENT

Authors, funders, patients, policymakers, and other stakeholders in research findings are legitimately concerned about dissemination of the results of controlled trials as quickly as possible. Several initiatives are underway to shorten the time interval between study completion to publication and from submission to publication. For example, many journals have established fast track policies for accelerating the publication process for studies in certain cases. At JAMA, peer reviewers are asked to return their comments within 24 to 48 hours for fast track reports and within 21 days for reports undergoing the standard review process. None of the articles in our study had been designated as fast track because JAMA's fast tract option was not initiated until late 1999.

In this study, we found no evidence that the time interval between manuscript submission and publication is influenced by study results, methodologic quality, or other study characteristics. Examination of factors leading to delays in publication have not tended to differentiate between the time authors take to write and submit study results from the time interval between submission and publication. Specifically, other analyses24 have included all studies undertaken, including those not published. Since negative findings are less likely to be published, analyses of time to publication that include unpublished reports focus on a different question than the one we addressed. Our study is the first to focus on published articles, the time interval between submission and publication, and possible time differences between reports describing positive and negative findings.

In a study of 66 completed randomized human immunodeficiency virus trials, time from study completion to publication was shorter for trials with positive results than it was for trials with negative results (a median of 1.7 years vs 3.0 years, respectively; P<.001).3 Trials with statistically significant results favoring the experimental arm (positive results) were submitted more quickly after completion than those with negative findings (median 1.0 vs 1.6 years, respectively; log rank, P = .001) and reached publication somewhat more quickly after submission (median 0.8 vs 1.1 years, respectively; P = .04). Of the 45 trials submitted for publication, 17 were rejected by 1 or more journals, and there was a nonsignificant tendency for positive results to be accepted more often on first submission (OR, 1.6; 95% CI; 0.5-5.6). Stern and colleagues also showed that clinical trials with null or nonsignificant results were slower to reach publication than those with significant findings (using the date of the ethics committee's approval as a start date).2 Misakian and Bero4 examined the number of years between start of funding and first publication for studies of passive smoking and found a statistically significant shorter interval for studies with statistically significant results compared with those with nonsignificant or mixed results.

We limited our definition of positive findings to those that were statistically significant for the primary outcome or the majority of outcomes examined when the primary outcome was unclear. Our results apply to a general population of trials submitted to JAMA and may not be applicable to other journals or trials in specific fields. We did not examine any discrete times such as the time taken by referees for review or by authors for revisions. Our study does not account for other features of journal articles that may influence time to publication, such as whether the results are clinically important or timely in some other way. It is possible but unlikely that our failure to observe any differences between groups in the time from submission to publication is attributable to a small sample size.

Conclusion

Among 133 published controlled trials, time to publication was not associated with statistical significance, methodologic quality, or other study characteristics.

REFERENCES

Dickersin K. How important is publication bias? a synthesis of available data.  AIDS Educ Prev1997;9(suppl A):15-21.
Stern JM, Simes RJ. Publication bias: evidence of delayed publication in a cohort study of clinical research projects.  BMJ.1997;315:640-645.
Ioannidis JP, Cappelleri JC, Sacks HS, Lau J. The relationship between study design, results, and reporting of randomized clinical trials of HIV infection.  Control Clin Trials.1997;18:431-44.
Misakian AL, Bero LA. Publication bias and research on passive smoking. comparison of published and unpublished studies  JAMA.1998;280:250-253.
Olson CM, Rennie D, Cook D, Dickersin K, Flanagin A, Hogan JW, Zhu Q, Reiling J, Pace B. Publication bias in editorial decision making.  JAMA.2002;287:2825-2828.

Figures

Figure. Time Interval Between Submission and Publication by Results
Graphic Jump Location
Kaplan Meier survival analysis P = .44.

Tables

References

Dickersin K. How important is publication bias? a synthesis of available data.  AIDS Educ Prev1997;9(suppl A):15-21.
Stern JM, Simes RJ. Publication bias: evidence of delayed publication in a cohort study of clinical research projects.  BMJ.1997;315:640-645.
Ioannidis JP, Cappelleri JC, Sacks HS, Lau J. The relationship between study design, results, and reporting of randomized clinical trials of HIV infection.  Control Clin Trials.1997;18:431-44.
Misakian AL, Bero LA. Publication bias and research on passive smoking. comparison of published and unpublished studies  JAMA.1998;280:250-253.
Olson CM, Rennie D, Cook D, Dickersin K, Flanagin A, Hogan JW, Zhu Q, Reiling J, Pace B. Publication bias in editorial decision making.  JAMA.2002;287:2825-2828.
CME
Also Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
Please click the checkbox indicating that you have read the full article in order to submit your answers.
Your answers have been saved for later.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.

Multimedia

Some tools below are only available to our subscribers or users with an online account.

788 Views
28 Citations
×

Related Content

Customize your page view by dragging & repositioning the boxes below.

Articles Related By Topic
Related Collections
Jobs
JAMAevidence.com

The Rational Clinical Examination: Evidence-Based Clinical Diagnosis
Quick Reference

The Rational Clinical Examination: Evidence-Based Clinical Diagnosis
Quick Reference