Workplace-Based Assessment of Residents’ Performance of Intra-Operative Pathology Consultations

Context: Competency-based medical education (CBME) is being implemented worldwide. In CMBE, residency training is designed around competencies required for unsupervised practice and use entrustable professional activities (EPAs) as workplace “units of assessment”. Well-designed workplace-based assessment (WBA) tools are required to document competence of trainees in authentic clinical environments. Objective: To develop a WBA instrument to assess residents’ performance of intraoperative pathology consultations and conduct a validity investigation. Design: The entrustment-aligned pathology assessment instrument for intraoperative consultations (EPA-IC) was developed through a national iterative consultation and used by clinical supervisors to assess residents’ performance at an Anatomical Pathology program. A psychometric analysis was conducted to provide information about the internal structure of the tool and focus groups explored aspects related to content, response process and consequences of assessment. Results: The content was considered appropriate, the assessment was feasible and acceptable by residents and supervisors, and had a positive educational impact by improving performance of intraoperative consultations and feedback to learners. The results had low reliability, which seemed to be related to rater biases, and supervisors were reluctant to fully entrust trainees due to cultural issues. Conclusion: We developed a WBA assessment instrument for assessing residents’ performance of intra-operative consultations. Given the formative nature of WBA, its use is justified by the positive educational impact and future studies should address the threats to validity. However, some of the threats seem to be embedded in the culture of pathology and one should approach WBA and CBME implementation as a continuous quality improvement project.