An introduction to the many-facet Rasch model as a method to improve observational quality measures with an application to measuring the teaching of emotion skills

Recent reviews have called attention to limitations of existing observational measures of early childhood classroom quality, including low rater agreement, ceiling/floor effects, and imprecise estimation. We offer an introduction to the many-facet Rasch model (MFRM), discussing how it can be used to iteratively improve measures to address such limitations. We provide an example of applying the MFRM to develop the EMOtion TEaching Rating Scale (EMOTERS) in order to capture the practices that teachers use in support of children's developing knowledge, expression, and regulation of emotion. The MFRM produced fine-grained statistics about how 23 raters scored 1609 10-minute video occasions from 18 classrooms with EMOTERS Version 6. Results were compared to traditional rater agreement and internal consistency statistics. With an eye toward continuous measure improvement, we discuss planned revisions of the EMOTERS as well as implications for the application of the MFRM to other measures.