Exploration of Validity Evidence for the P MEX in a Residency Admissions Process

The residency admissions process is a high-stakes assessment system with the purpose of identifying applicants who best meet standards of the residency program and the medical specialty. With the growing importance of professionalism in medical education, it is imperative to develop professionalism assessments that demonstrate robust validity evidence. The Professionalism Mini-Evaluation Exercise (P-MEX) is an assessment that has demonstrated validity evidence in the authentic clinical setting. This dissertation examines validity evidence for the P-MEX, a focused and standardized assessment of professionalism, in a simulated patient setting. Ginsburg’s professionalism framework and Messick’s unified validity framework were used to frame this study. This study examines the internal structure and relations to other variables validity evidence for the P-MEX using longitudinal data from eight cohorts of pediatric applicants from admission to the end of the first year of post-graduate training. Results indicate that the P-MEX demonstrates moderate reliability and predicts professionalism performance at the end of the first year of training. The P-MEX also demonstrates a four-factor model of: doctor-patient relationship skills, interprofessional skills, professional demeanor, and reflective skills. Identifying the factorial structure of the P-MEX allows for the provision of diagnostic and actionable professionalism feedback. Validity evidence supports the use of P-MEX scores as part of the admissions process to assess professionalism, as part of the admissions process. Educators may wish to integrate the P-MEX assessment into their professionalism curricula.