A Model for Improving Survey Outcomes by Reducing Cognitive Load

Surveys are a common method for collecting information. However, when practitioners (who are usually not survey design experts) seek to create surveys that follow the myriad recommendations and best practices in the wide-ranging survey design literature, they face a daunting task. Currently, there is no theory-driven, practical model that practitioners can use to design surveys. In this research, I used cognitive load theory to construct a model that practitioners could employ to develop surveys. Cognitive load refers to the amount of mental effort, or thinking, required to respond to survey items. The purposes of my two studies were to create a model for reducing cognitive load in survey items and instruments and then test the efficacy of a portion of that model. In my first study, I created an online survey with two different versions of each item that were parallel in content but differed in their theoretical cognitive loads. Students (n = 64) identified the version of each item that they felt required more mental effort to respond to. In a second study, I randomly assigned students to complete a survey that contained either all the high cognitive load (HCL) versions of the items (n = 280 students) or all the low cognitive load (LCL) versions of the items (n = 277 students). I calculated the response rate for each survey and for items on a survey, the time students took to respond to each survey, and Rasch student fit statistics and point-measure correlations to detect response sets in the students’ ratings. Students reported that the HCL versions of the items required more mental effort to respond to than the LCL versions. They spent significantly more time responding to items on the HCL survey than to items on the LCL survey, and they skipped more items on the HCL survey than on the LCL survey. Finally, while the number of students who displayed response patterns indicative of response set use was similar for those answering items on both surveys, the types of aberrant response patterns that the two groups exhibited differed.