Integrating Direct Observations in Clinical Teachers' Workflow: An Exploratory Cognitive Task Analysis

Background: Successful implementation of competency-based assessments is largely contingent on Direct Observation (DO). However, its uptake remains problematic and threatens its educational mandate. DO-focused research has primarily been examined from a psychometric and sociocultural perspective. Cognitive Task Analysis (CTA) provides tools to elicit knowledge from experienced users and understand their performance in complex environments, which is then used to design tools to enhance task performance. The purpose of this study was to understand, using CTA, the cognitive functions and cues that drive CTs’ decision-making in integrating DO in their workflow while balancing the competing demands of simultaneously being an educator and healthcare provider. Methods: We conducted an qualitative study guided by Applied Cognitive Task Analysis (ACTA). We interviewed CTs in internal medicine-based specialties who work on inpatient medical units at a single academic institution. CTs were purposively sampled following a nomination process to find participants who excel at using DO in inpatient settings. Data were analyzed using reflexive thematic analysis. Results: Six CTs with a median of 8 (range 2-13) years of experience participated and described a dual responsibility in attending to patients’ and learners’ needs, with patient safety ultimately prioritized as the key driver for DO. Participants shared three cognitive functions underlying their work on inpatient units and describing the judgements and decisions needed for DO to occur: (1) making tasks count twice to promote efficient use of DO, (2) planning and re-planning DO as clinical activities unfold, and (3) self-reflections that mold educational practices that integrate DO. CTs described multiple cues that inform the need for DO and categorized them in relation to (1) patients and their care, (2) learners, (3) teachers’ experiences and (4) the learning environment. Discussion: ACTA was a useful and systematic approach to tap into the expertise of a select group of CTs. The insight gained on patient safety as the primary cognitive driver for DO provides an opportunity to shift faculty development initiatives. Using the identified cognitive functions and cues, we discuss potential opportunities for faculty development to better prepare and engage CTs in integrating DO in their workflow.