Formative Fugues: Conceptualizing Data-Driven Formative Feedback for Open-Ended Learning Environments

Interactive digital environments provide learners with opportunities to collaboratively explore complex, open-ended problem spaces.To support learners’ exploration of complex problems, I reconceptualized the nature of formative feedback for open-ended complex learning environments, shifting the focus from learner centric to problem-space centric feedback. This feedback is analogous to “fugues” (short reusable exploration paths, explaining semantics of actions) as defined by Reitman (1965) for musical compositions (Mallavarapu et al., 2020). Traditional techniques for generating formative feedback don’t scale well to environments that have a multiplicity of possible (a) goals, (b) solution strategies, and (c) solution strategy paths (Le et al., 2013). By bootstrapping on the explorations of prior learners, I propose a novel data-driven approach to produce situationally relevant formative feedback for learners, formative fugues, that is easily extendable - the formative feedback library grows as learners interact. And because the emphasis is on the problem space rather than an individual learner’s cognition, this method readily supports collaborative problem solving. To generate this feedback, the approach employs causal modelling to learn micro-level patterns and then uses a pattern matching algorithm to mine these patterns, exposing exploration paths of prior learners who have used the learning environment (data collected under IRB approved protocol). The formative fugues do not presume to know the goals of the learner, but have the potential to help the learners understand what choices and events led to the current state of the problem space, and what paths forward are possible. When computational outputs are intertwined with human practices, it is important to attend to the intertwined dependencies by engaging with the stakeholders to validate the interpretability and utility of the outputs. Specifically I (1) engage a domain expert to discern the validity of the information being presented (the what), and (2) engage a pedagogical expert to envision how the provided information intersects with practice (the how). These sessions, (1) helped identify the alignment of the fugues with the educational goals of the learning environment, (2) commented on the interpretability of the information by identifying practices afforded by fugues and (3) suggested recommendations for delivery of the fugues.