Context.— All authors may not be equal in the eyes of reviewers. Specifically,
well-known authors may receive less objective (poorer quality) reviews. One
study at a single journal found a small improvement in review quality when
reviewers were masked to author identity.
Objectives.— To determine whether masking reviewers to author identity is generally
associated with higher quality of review at biomedical journals, and to determine
the success of routine masking techniques.
Design and Setting.— A randomized controlled trial performed on external reviews of manuscripts
submitted to Annals of Emergency Medicine, Annals of Internal
Medicine, JAMA , Obstetrics & Gynecology , and Ophthalmology .
Interventions.— Two peers reviewed each manuscript. In one study arm, both peer reviewers
received the manuscript according to usual masking practice. In the other
arm, one reviewer was randomized to receive a manuscript with author identity
masked, and the other reviewer received an unmasked manuscript.
Main Outcome Measure.— Review quality on a 5-point Likert scale as judged by manuscript author
and editor. A difference of 0.5 or greater was considered important.
Results.— A total of 118 manuscripts were randomized, 26 to usual practice and
92 to intervention. In the intervention arm, editor quality assessment was
complete for 77 (84%) of 92 manuscripts. Author quality assessment was complete
on 40 (54%) of 74 manuscripts. Authors and editors perceived no significant
difference in quality between masked (mean difference, 0.1; 95% confidence
interval [CI], −0.2 to 0.4) and unmasked (mean difference, −0.1;
95% CI, −0.5 to 0.4) reviews. We also found no difference in the degree
to which the review influenced the editorial decision (mean difference, −0.1;
95% CI,−0.3 to 0.3). Masking was often unsuccessful (overall, 68% successfully
masked; 95% CI, 58%-77%), although 1 journal had significantly better masking
success than others (90% successfully masked; 95% CI, 73%-98%). Manuscripts
by generally known authors were less likely to be successfully masked (odds
ratio, 0.3; 95% CI, 0.1-0.8). When analysis was restricted to manuscripts
that were successfully masked, review quality as assessed by editors and authors
still did not differ.
Conclusions.— Masking reviewers to author identity as commonly practiced does not
improve quality of reviews. Since manuscripts of well-known authors are more
difficult to mask, and those manuscripts may be more likely to benefit from
masking, the inability to mask reviewers to the identity of well-known authors
may have contributed to the lack of effect.