Human-Centric Physiological Approaches to Improve User Interaction and Adaptation in Human-Robot Teaming

Human-robot interaction (HRI) is a rapidly evolving field with a diverse range of applications, including manufacturing, aviation, surgery, rehabilitation, education, rescue, and military operations. As humans are key stakeholders or end users in these scenarios, there is a major need for research on human interaction aspects and user experience in the design and development of robots. Robots are expected to be aware of human preferences and internal states (via physiological signals) and adapt their behavior accordingly, so that they develop a symbiotic relationship with the human and maximize the beneficial outcomes of HRI (Gervasi et al., 2022, 2024). This dissertation comprises six projects which are focussed on two types of human-robot teaming tasks: (i) shared workspace tasks, where the human is an operator and (ii) physical augmentation tasks, where the human is a user. The research focus is on understanding human internal states (such as adaptation, fluency, effort, comfort), representation of internal states using psychophysiological measures and closing the human-robot loop to enhance user experience. Shared workspace tasks (Human as operator): Fluency, referred to as a well-synchronized meshing of actions between human and robot, is an important factor that affects team performance and long-term sustained collaboration at the workplace. Chapter 2 investigates the potential for Electrocardiogram (ECG) metrics to indicate the fluency of human-robot teaming in shared space collaboration. The results showed that fluency in HRC tasks correlated with specific HRV features, enabling objective fluency prediction using machine learning models with useful accuracy. In manufacturing scenarios, automated robotic systems require varying degrees of supervisory control by a human. However, there is limited research on individual differences in worker preferences regarding the level of automation and decision authority, showing the need for more human-centric studies in human-robot collaboration tasks (Weiss et al., 2021). More specifically, physiological responses representative of changes in mental effort during different levels of automation in collaborative manufacturing have not been studied. Chapter 3 bridges this research gap. Findings reveal that individual differences exist in preferences for robot automation levels, with many favoring a medium automation level that includes human decision authority. Nonlinear ECG-derived heart rate variability (HRV) metrics reveal an increase in mental effort with lower automation, suggesting that it has applications in human-aware robotic interfaces. In addition to worker preferences and teaming fluency, adequate operator support is of crucial importance to enable the successful adoption of robots in the workplace. While Augmented Reality (AR) user interfaces have been found to improve engagement and user experience in educational settings, their effectiveness for human-robot interaction needs to be evaluated. Chapters 4 and 5 investigate this research direction. For robot control and operation, physical touch-based user interfaces such as the electronic tablet were perceived as more usable compared to AR based interfaces, indicating ergonomic improvements for future adoption. Physical augmentation tasks (Human as user): In wearable robotics, particularly for lower limb exoskeletons that assist with movements such as walking and squatting, the human-in-the-loop (HIL) optimization method has been proven to be successful in reducing physical effort by delivering personalised assistance to each user (Ding et al., 2018; Kantharaju et al., 2022, 2023; J. Zhang et al., 2017). However, the current method of optimization is based on an energy cost function estimated using a respiratory system, which is uncomfortable, non-portable, produces noisy measurements and requires a long estimation time. These issues limit the application of human-in-the-loop exoskeleton optimization method in the real world. In Chapter 6, I address this gap by finding an alternate cost function based on users’ foot center of pressure that is faster, comfortable, portable, and reduces injury risk in exoskeleton-assisted squatting. Another limitation of the current state-of-the-art HIL optimization method is that it optimizes for only user effort and does not explicitly incorporate the human’s movement adaptation to the assistance from the device, which is important as adaptation due to training has been found to enhance the device’s performance and user outcomes (M. Kim et al., 2022; Poggensee & Collins, 2021). In Chapter 7, I show how heart rate complexity, measured using a wearable ECG sensor, can be used to assess a user’s adaptation status to assistive devices, based on its changes before and after training users to walk with a hip exosuit. Chapter 8 deals with the implementation and evaluation of a dual-objective human-in-the-loop optimization scheme that optimizes the assistance from a lower limb exoskeleton based on both energy expenditure as well as the user’s movement adaptation (gait symmetry) to robotic assistance. The results of this study show that the incorporation of an implicit coaching reward within a dual objective optimization method yields optimal assistance that is both energy efficient as well as user-preferred, showcasing its potential for enabling natural human-exoskeleton interaction and supporting exoskeleton adoption in the real world.