Mixed Effects Logistic Regression to Inform Component Weighting for a Graduation Competency Exam

This study examines the relationship between medical student performance on a local Graduation Competency Examination (GCE) and national United States Medical Licensing Examination (USMLE) Step 2 Clinical Skills (CS) pass-fail results. An investigation was carried out to determine ideal weighting of GCE components that maximize predictive association with USMLE Step 2 CS pass-fail results. The performance of 1,056 students over 6 academic years on both GCE and USMLE Step 2 CS was analyzed. Pearson correlation coefficients were calculated between GCE components and USMLE Step 2 CS pass-fail results. GCE Patient Note (PN) performance and GCE Communication and Interpersonal Skills (CIS) performance were significantly associated with USMLE Step 2 CS pass-fail result. GCE Physical Exam (PE) performance was not significantly associated with USMLE Step 2 CS pass-fail results. Mixed effects logistic regression models were estimated with USMLE Step 2 CS pass-fail results as outcome, each individual GCE component as fixed effect and year as random effect. Better performance on each of PE, PN and CIS individually was associated with increased odds of USMLE Step 2 CS pass results. A mixed effects logistic regression model was estimated with USMLE Step 2 CS pass-fail results as outcome, all GCE components as fixed effects and year as random effect. Better performance on PN was associated with increased odds of USMLE Step 2 CS pass results. Mixed effects logistic regression models were estimated with USMLE Step 2 CS pass-fail result as outcome, GCE Integrated Clinical Encounter (ICE) performance as fixed effect and year as random effect. Models were estimated using various PE and PN weighting combinations to comprise ICE score (e.g., 100%PE/0%PN, 90%PE/10%PN, 80%PE/20%PN). For all weighting combinations, better performance on ICE was associated with increased odds of USMLE Step 2 CS pass results. Maximum odds were obtained with weighting 20%PE/80%PN.