Measuring Mindsets in Medical Students: Validity of the Implicit Theories of Intelligence Self-Scale

Background Mindsets are a theory of motivation that holds promise in helping medical education achieve its vision of developing self-regulated lifelong learners. However, minimal mindset research has been conducted within medical education. Compared to previously studied learners, medical students are a distinct population within a unique educational context. The primary purpose of this study is to conduct psychometric testing on the implicit theories of intelligence self-theory scale (ITIS-S) to measure mindsets in medical students. Methods In January 2021, medical students enrolled at Vanderbilt University School of Medicine were invited to complete a 57-question online survey which included the 8-item ITIS-S measured on a 6-point Likert-type scale. The ITIS-S retest was administered 2 weeks later. Measures of goal orientation, adaptability, engagement, and epistemological beliefs were also included. Internal structure was assessed with Cronbach’s alpha and confirmatory factor analysis (CFA) with maximum likelihood ratios. Response process was assessed with test-retest reliability as measured by Pearson’s correlation coefficient and two-tailed paired t-tests. Pearson’s correlation coefficient was used to determine the relationship to other variables. Multiple linear regression was used to determine the impact of demographic factors on ITIS-S score. Results The response rates were 36.8% (168/456) for the initial survey and 55.4% (93/168) for the retest. Participants did not differ significantly from non-participants with respect to gender [45.2% (76/168) vs. 52.8% (152/288) male, p=0.12], underrepresented in medicine (URiM) status [16.7% (28/168) vs. 21.9% (63/288) identify as URiM, p=0.18], or age [26.1 (2.8) vs. 25.8 (3), p=0.46]. The mean (SD) score on the ITIS-S was 33.6 (8.2) and Cronbach’s α was 0.96. The retest reliability was strong (r=0.72, p<0.001) and the ITIS-S means (SD) did not differ significantly on retest [32.6 (8.4) vs. 32.4 (8.6), p=0.72]. CFA indicated a good fit of the two-factor model (χ2 21.469, df=19, p=0.311; GFI 0.97; RMSEA 0.03). ITIS-S score had a weak but statistically significant correlation with goal orientation (r=0.24, p<0.01), adaptability (r=0.22, p<0.01), and engagement (r=0.17, p<0.05). Age, gender, URiM status, and phase of medical school did not statistically significantly predict ITIS-S score [F(4, 92) = 1.27, p=0.29, R2=0.06]. Discussion The ITIS-S demonstrated a high degree of internal consistency and strong retest reliability with a good fit to the theorized two-factor model. Although correlations between mindset and goal orientation, adaptability, and engagement were weak, they were statistically significant and consistent with hypothesized relationships. Demographic factors did not significantly impact ITIS-S scores. Conclusion This study provides validity evidence for the use of the Implicit Theories of Intelligence Self-Theory Scale for use in medical students and is foundational to a deeper exploration of motivation in health professions education, especially in the era of competency-based medical education.