Feasibility And Validity Evidence for Use of Multimedia Vignettes in a Year One Neuroscience Examination

Multiple-choice questions using multimedia clips of clinical phenomena can provide reliable assessments in medical school clerkships and licensing examinations. The purposes of this study were to develop a test, using multimedia clips for a pre-clerkship course and to demonstrate its validity, feasibility and acceptability. A 50 question computer-based test with 25 multimedia vignettes (2 questions/vignette) was developed and administered to 50 medical students at the conclusion of a Year One neuroscience block (course). Each vignette portrayed a patient’s findings or history; questions were blueprinted to course content with a focus on neuroanatomy/localization, diagnosis, neurophysiology, neurochemistry, neuropharmacology or neuropathology. Seven poorly performing items were deleted in the final scoring. The mean raw score on the remaining 43 items was 37.92 (88%) with a standard deviation of 3.34 (7.7%). Cronbach’s alpha was 0.66. There was a moderate correlation between this test and the standard 100-item multiple choice question final examination (Pearson’s correlation coefficient=0.63). Forty-seven of fifty (94%) students responded to a survey regarding the test and their responses were analyzed using qualitative methods to identify themes. Students overwhelmingly indicated that the multimedia questions more realistically reflected the clinical context in which they will function as physicians and were appropriate for assessment. Faculty members agreed that the test was appropriate for student assessment. Validity evidence, along with student and faculty perceptions of acceptability, supports using multimedia-based items as an assessment component in a neuroscience course. The results of this study support the further development of multimedia vignettes for assessments in preclinical courses.