Tools to Assess Clinical Skills of Medical Trainees

To the Editor: Dr Kogan and colleagues¹ reported the results of a systematic review of observation tools used to assess clinical skills of medical trainees, summarizing the evidence of the tools' validity and outcomes. The authors stated in their methods that every article was independently abstracted by 2 authors and that differences in data abstraction were resolved through consensus adjudication.

Regarding the differences in data abstraction, I would like to know what was the proportion of differences that required adjudication and if those differences were related to specific items of the abstraction form. In their review, the authors did not provide quantitative measures of the interrater reliability; reporting κ coefficients for the main review items would have been informative. I would also like to know what the authors meant by consensus adjudication (methods and number of persons involved).