0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Research Letter |

Study Design, Publication Outcome, and Funding of Research Presented at International Congresses on Peer Review and Biomedical Publication FREE

Mario Malički, MD, MA1; Erik von Elm, MD, MSc2; Ana Marušić, MD, PhD1
[+] Author Affiliations
1Department of Research in Biomedicine and Health, University of Split School of Medicine, Split, Croatia
2Institute for Social and Preventive Medicine, University Hospital Lausanne, Lausanne, Switzerland
JAMA. 2014;311(10):1065-1067. doi:10.1001/jama.2014.143.
Text Size: A A A
Published online

The first International Congress on Peer Review and Biomedical Publication (PRC) was organized in 1989 to “subject the editorial review process to some of the rigorous scrutiny that editors and reviewers demand of the scientists whose work they are assessing.”1 Since then, peer review research was introduced as a Medical Subject Heading (MeSH),2 and the number of indexed articles has been continuously increasing. To evaluate the development of peer review research in biomedicine, we analyzed research presented at all PRCs since 1989.

We established a retrospective cohort of PRC poster and podium abstracts and used author names to search the PRC’s website, Web of Science (WoS), and PubMed for full articles through August 20, 2013. We collected data on authorship, time to publication, declared funding sources, article availability, and citation counts in WoS. Two authors classified study design independently, with acceptable agreement (κ = 0.78). Data were analyzed using MedCalc; 2-sided significance testing included χ2 tests and Kruskal-Wallis test (type I error was set to P < .05).

Of 614 presented abstracts, 75% described observational studies; 18%, intervention studies, and 7%, opinion pieces (Table 1). Over time, the number of opinion pieces decreased from 17 in 1989 to 1 in 2013 (χ21 for trend = 47.3, P < .001). The number of cohort studies increased from 0 in 1989 to 8 in 2013 (χ21 = 10.7, P = .002). Feasibility studies increased from 1 in 1989 to 20 in 2013 (χ21 = 11.3, P < .001). The median number of abstract authors increased from 1 (95% CI, 1-1) in 1989 to 4 (95% CI, 4-5) in 2013 (P < .001). Of the 504 abstract presentations from the first 6 PRCs, 305 (61%) led to 294 published articles (Table 2). From abstract presentation to publication, there were no changes in the byline order or number of byline authors in 166 abstracts (56%), whereas 83 abstracts (28%) had changes in the number of authors listed and 45 (15%) had changes in the byline order. One hundred fourteen articles (38%) were published in JAMA, 21 (7%) in BMJ, 12 (4%) in Annals of Emergency Medicine, and 8 (3%) each in the Journal of Clinical Epidemiology and in PLoS ONE (Table 2). The median time to publication was 14 months (95% CI, 12-16), when excluding 110 articles in JAMA theme issues. One hundred articles (63%) were freely available online.

Table Graphic Jump LocationTable 1.  Study Designs, Publication Proportion, and Funding Sources of Research Presented at International Congresses on Peer Review and Biomedical Publication (PRC), 1989-2013
Table Graphic Jump LocationTable 2.  Publication Output, Country of Origin, Declared Funding and Subsequent Citations of Studies Presented at International Congresses on Peer Review and Biomedical Publication, 1989-2013

Funding was reported in 106 (36%) published articles that had been presented as abstracts at the 1989-2009 PRCs and in 45 abstracts (41%) presented at the 2013 PRC, most commonly from public or charity sources (Table 1). The absolute number and proportion of articles with declared funding increased over time, with a peak in 2005 (Table 2).

Two hundred eighty-four published articles (97%) were indexed in WoS; 265 (93%) of them received at least 1 citation, with a median of 20 (95% CI, 17-27) citations per article. Articles with the most citations were on a reporting guideline for health research3 (published in 17 journals; n = 1798 citations), synthesis of evidence4 (n = 1016), and publication bias5 (n = 547).

Peer review research uses various study designs and is published in a broad spectrum of journals. However, experimental studies aimed at improving methods of peer review and reporting of biomedical research are still underrepresented. Although the peer review research community is aware of the consequences of nonpublication of research,6 39% of studies presented at PRCs have not been fully published. In our cohort, we were unable to determine whether the underreporting was selective (eg, favoring positive results) and were not able to determine its causes. Lack of suitable journal outlets is an unlikely explanation because there was no decrease in publication output after JAMA ceased its PRC theme issues in 2005. Because our cohort represents research presented for more than 20 years at the discipline’s major meeting, it may have limited generalizability to research presented elsewhere.

Peer review and other editorial procedures have the potential to significantly influence the knowledge base of health care. Despite their critical role in biomedical publishing, methods of peer review are still underresearched and lack dedicated funding. Systematic and competitive funding schemes are needed to build and sustain excellence, innovation, and methodological rigor in peer review research.

Section Editor: Jody W. Zylke, MD, Senior Editor.

Corresponding Author: Ana Marušić, MD, PhD, Department of Research in Biomedicine and Health, University of Split School of Medicine, Šoltanska 2, 21000 Split, Croatia (ana.marusic@mefst.hr).

Author Contributions: Dr Marušić had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: von Elm, Marušić.

Acquisition of data: all authors.

Analysis and interpretation of data: all authors.

Drafting of the manuscript: all authors.

Critical revision of the manuscript for important intellectual content: all authors.

Statistical analysis: Malički.

Administrative, technical, and material support: Malički, Marušić.

Study supervision: von Elm, Marušić.

Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Drs Marušić and von Elm are members of the Advisory Board of the Seventh International Congress on Peer Review and Biomedical Publication.

Previous Presentation: This study was presented at the Seventh International Congress on Peer Review and Biomedical Publication, Chicago, Illinois, September 8-10, 2013.

Additional Contributions: We thank the Esteve Foundation for organizing a Discussion Group on Editorial Research in Barcelona (Catalonia, Spain) on December 12-13, 2012, which gave us the idea for the study. We thank Annette Flanagin, RN, MA, and Drummond Rennie, MD, at JAMA for providing access to the abstracts and JAMA articles from the early Peer Review Congresses, for which they were not compensated.

Rennie  D. Preface. In: Peer Review in Scientific Publishing. Papers from the First International Congress on Peer Review and Biomedical Publication. Chicago, IL: Council of Biology Editors; 1991.
National Library of Medicine. Medical Subject Headings: “Peer Review, Research.”http://www.ncbi.nlm.nih.gov/mesh/?term=peer+review%2C+research. Accessed October 5, 2013.
Bossuyt  PM, Reitsma  JB, Bruns  DE,  et al; Standards for Reporting of Diagnostic Accuracy.  Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. Clin Chem. 2003;49(1):1-6.
PubMed   |  Link to Article
Dickersin  K, Scherer  R, Lefebvre  C.  Identifying relevant studies for systematic reviews. BMJ. 1994;309(6964):1286-1291.
PubMed   |  Link to Article
Dickersin  K.  The existence of publication bias and risk factors for its occurrence. JAMA. 1990;263(10):1385-1389.
PubMed   |  Link to Article
Scherer  RW, Langenberg  P, von Elm  E.  Full publication of results initially presented in abstracts. Cochrane Database Syst Rev. 2007;(2):MR000005.
PubMed

Figures

Tables

Table Graphic Jump LocationTable 1.  Study Designs, Publication Proportion, and Funding Sources of Research Presented at International Congresses on Peer Review and Biomedical Publication (PRC), 1989-2013
Table Graphic Jump LocationTable 2.  Publication Output, Country of Origin, Declared Funding and Subsequent Citations of Studies Presented at International Congresses on Peer Review and Biomedical Publication, 1989-2013

References

Rennie  D. Preface. In: Peer Review in Scientific Publishing. Papers from the First International Congress on Peer Review and Biomedical Publication. Chicago, IL: Council of Biology Editors; 1991.
National Library of Medicine. Medical Subject Headings: “Peer Review, Research.”http://www.ncbi.nlm.nih.gov/mesh/?term=peer+review%2C+research. Accessed October 5, 2013.
Bossuyt  PM, Reitsma  JB, Bruns  DE,  et al; Standards for Reporting of Diagnostic Accuracy.  Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. Clin Chem. 2003;49(1):1-6.
PubMed   |  Link to Article
Dickersin  K, Scherer  R, Lefebvre  C.  Identifying relevant studies for systematic reviews. BMJ. 1994;309(6964):1286-1291.
PubMed   |  Link to Article
Dickersin  K.  The existence of publication bias and risk factors for its occurrence. JAMA. 1990;263(10):1385-1389.
PubMed   |  Link to Article
Scherer  RW, Langenberg  P, von Elm  E.  Full publication of results initially presented in abstracts. Cochrane Database Syst Rev. 2007;(2):MR000005.
PubMed
CME
Also Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
Please click the checkbox indicating that you have read the full article in order to submit your answers.
Your answers have been saved for later.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.

Multimedia

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging & repositioning the boxes below.

See Also...
Articles Related By Topic
Related Collections
PubMed Articles