Workplace-Based Assessment Using the Pediatric Critical Care Medicine Entrustable Professional Activities

A validation study of a novel workplace-based assessment tool was conducted using a mixed-methods approach. Messick’s conceptual framework was used to determine validity of this novel, center-specific assessment tool of Pediatric Critical Care Fellows. Specific evidence was sought to provide content validity, relationship to other variables validity, internal structure validity, and response process validity. The frequency of assessment topics was correlated with the most common professional activities performed within the PICU providing content and relationship to other variables validity evidence. Evaluation of the influence of acuity and fellowship year of training provided internal structure validity evidence. Understanding the reasons behind the assignment of the entrustment level by faculty, and the consistency with previous reports of entrustment influences in the literature, provided cognitive response process validity. Finally, compliance rates provided evidence of the physical response process validity. Of the 21 topics, 95% were assessed at least once during the academic year with 70% assessed at least five times. Additionally, the most frequently assessed topics correlated with the most common professional activities in the intensive care unit. When assessed across the same acuity level for this sub-EPA, first year fellows demonstrated a mean entrustment level of 2.38 (+0.51) compared to 3.14 (+0.38) in second year fellows, and 3.86 (+0.90) in third year fellows. All of the common literature-referenced influences of entrustment were also cited by faculty as the reasons behind the assigned entrustment level. Faculty were 75%+20% compliant (range 50-100%) with twice weekly completion of the EPA-based WPBA assessment tool while fellows demonstrated more variability with 87%+46% compliant (range 28-100%) compliance.