Mobile Interface Development for Tongue-Controlled Devices Aimed at Training and Real-Life Interaction

Allowing individuals with disabilities to the upper limbs to have access to smartphones is an ongoing challenge. One solution, developed in the Wearable Technology and Sensory Enhancement laboratory, comprises of Bluetooth-enabled assistive devices to be placed discreetly inside the oral cavity. For this work, two devices have been adopted: one button-based and one trackpad-based. The devices are positioned on the palate, encased in a discreet dental retainer, and controlled by the tongue. This work presents a custom-made application to be paired with the two tongue-controlled assistive devices. The application aims at allowing simple interaction by the user, who is enabled to perform a plethora of different functionalities, identified through a survey distributed among disabled individuals. The most requested functionalities are access to 911 calls, interaction with keyboards and communication means, and control of a wheelchair, which have been implemented. To develop the needed interfaces, the framework React Native was adopted, due to its cross-platform compatibility and the high flexibility it allows. The application was then tested on an Android phone, to assess its performance. Keystroke Level Model analysis of the main functionalities was executed, demonstrating the theoretical usability of the interfaces. Interaction by an expert user further tested the functioning of the application, comparing it to the default environment of the phone. This confirmed the hypothesized simplifications estimated by applying Fitts Law’s principle to the design of custom-made components, such as a keyboard. A tongue training environment was included in the application’s development, aiming to increase the strength and precision of the tongue and ease interaction with the assistive technology placed on the palate. The environment is made up of six games, each focusing on a specific movement required by the user. The interfaces, jointly with the assistive devices, could represent a solution to close the technological gap that involves people with injuries that paralyze the upper limbs.