Validity Evidence for a Medical Student Assessment Tool in Knowledge of Operating Room Sterile Technique

Purpose: The purpose of this study is to report on the validity evidence for a scrub training knowledge assessment tool to demonstrate the utility and robustness of a multimodal, EPA-aligned, mastery learning scrub training curriculum which increases entrustment in scrub training techniques. Method: In 2017, a multimodal, EPA-aligned, mastery learning curriculum was implemented at a single academic medical institution. The 25 selected-response knowledge assessment tool content was anchored with our curricular objectives and operating room policies. We collected validity evidence during the 2017-2018 academic year. Results: 220 medical students and physician assistant students participated in the scrub training curriculum from April 2017 to June 2018. The mean pre-curriculum knowledge assessment score was 74.4% (SD= 15.6) and 90.1% (SD= 8.3) for the post-curriculum knowledge assessment, yielding a Cohen’s d= 1.06, P < 0.001. The mean item discrimination index was 0.36. The mean item difficulty was 0.74. The internal-consistency reliability was α= 0.71. Students with previous scrub training scored significantly better on the pre-curriculum knowledge assessment than those without previous scrub training (81.9 (SD= 12.6) vs. 67.0 (SD= 14.9); P <0.001). The cut score of 92% was determined using a Mastery Learning Angoff. Conclusions: This study describes the development, administration, and validity evidence of a multimodal, EPA-aligned, mastery-based curriculum and assessment tool for scrub training. We support the use of scores derived from this test for assessing scrub training knowledge among medical students and physician assistant students.