A multi-source feedback.pdf (264.02 kB)
Download file

A multi-source feedback tool for measuring a subset of Pediatrics Milestones

Download (264.02 kB)
journal contribution
posted on 18.11.2017, 00:00 by Alan Schwartz, Melissa Margolis, Sara Multerer, Hilary Haftel, Daniel J. Schumacher
Background: The Pediatrics Milestones Assessment Pilot employed a new multisource feedback (MSF) instrument to assess nine Pediatrics Milestones among interns and subinterns in the inpatient context. Objective: To report validity evidence for the MSF tool for informing milestone classification decisions. Methods: We obtained MSF instruments by different raters per learner per rotation. We present evidence for validity based on the unified validity framework. Results: One hundred and ninety two interns and 41 subinterns at 18 Pediatrics residency programs received a total of 1084 MSF forms from faculty (40%), senior residents (34%), nurses (22%), and other staff (4%). Variance in ratings was associated primarily with rater (32%) and learner (22%). The milestone factor structure fit data better than simpler structures. In domains except professionalism, ratings by nurses were significantly lower than those by faculty and ratings by other staff were significantly higher. Ratings were higher when the rater observed the learner for longer periods and had a positive global opinion of the learner. Ratings of interns and subinterns did not differ, except for ratings by senior residents. MSF-based scales correlated with summative milestone scores. Conclusion: We obtain moderately reliable MSF ratings of interns and subinterns in the inpatient context to inform some milestone assignments.


Publisher Statement

Post print version of article may differ from published version. The definitive version is available through Taylor and Francis Ltd at DOI:10.3109/0142159X.2016.1147646


Taylor and Francis Ltd



Issue date


Usage metrics