Dynamic Environment-based Visual User Interface System for Intuitive Navigation Target Selection for Brain-actuated Wheelchairs

Abstract

Visual user interface paradigms are one of the key modules for brain-actuated wheelchairs, which are a class of promising assistive devices that can increase the autonomy and mobility of people suffering from severe motor impairments. This work proposes a Dynamic Environment-based Visual Interface System (DEVIS) for intuitive navigation target selection for brain-actuated wheelchairs. It is composed of a novel Dynamic Visual Interface (DVI), an RGB image-based perception module (indoor scene classification, object detection and classification, and object tracking), and a P300-based BCI. The DVI displays potential navigation goals in three forms of visual cues: an RGB camera image streaming with object bounding boxes overlaid on objects detected and tracked, three global points of interest (indoor places), and two static commands. Hence, the DVI allows a user to select, through the P300-based BCI, navigation targets/commands that are flashing independently and randomly to create an oddball paradigm. Experimental evaluations were carried out in a dynamic setting with 5 participants, who were asked to select targets displayed in the DVI. The dynamic setting was obtained using RGB image sequences of the ISR-RGB-D Dataset, which represents a mission performed by a mobile robotic platform. The target selection was performed through a P300-based BCI, using non-self-paced and self-paced modalities. The obtained results show that the proposed DEVIS can run in real-time and that users are able to correctly select targets with high BCI accuracy rates.