Supporting Older Adults Navigate Feature-rich Mobile UIs with Voice Input

While mobile device ownership among older adults has experienced a global surge, older adults tend to lag behind their younger counterparts both in the number of mobile applications they use and the number of features they engage with on these applications. Mobile user interfaces (UIs) often present a challenge for older users due to the abundance of features squeezed onto small screens, coupled with frequent updates that alter visual layouts. One common approach to address this interaction issue is to design senior-friendly applications with simpler interfaces and fewer functionalities. However, such simplifications can actively stigmatize older users and act as a barrier to intergenerational experiences. This dissertation delves into the way of supporting older users in navigating feature-rich mobile applications. It provides a comprehensive analysis of the interaction challenges faced by older users with mobile UIs and discusses the design and development of an assistive tool to aid them in navigating these complex applications. Initial exploration of interaction challenges revealed that non-motor navigational issues were more difficult to address than motor issues, often leading to frustration and resignation. One specific challenge identified was the difficulty in quickly locating relevant UI features. In response, a design solution was devised to assist users in locating UI features or elements through verbal queries. During the design process, the way users articulate interaction issues verbally was explored, leading to the identification of five query types. Furthermore, a design exploration of visual cues to communicate a reduced set of UI features to older adults was conducted, resulting in the determination of the most effective visual cues for conveying information to older users. It helped older adults find a specific feature more quickly and more accurately — on par with younger adults. Based on these experiments, a new interaction technique was proposed: Nav(igation) Nudge. 'Nav Nudge' uses large language models and voice interactions to reduce the feature search space of a feature-rich UI on demand.