Efficient Visual Perception of Human-Robot Walking Environments using Semi-Supervised Learning

Abstract

Abstract Convolutional neural networks trained using supervised learning can improve visual perception for human-robot walking. These advances have been possible due to large-scale datasets like ExoNet and StairNet - the largest open-source image datasets of real-world walking environments. However, these datasets require vast amounts of manually annotated data, the development of which is time consuming and labor intensive. Here we present a novel semi-supervised learning system (ExoNet-SSL) that uses over 1.2 million unlabelled images from ExoNet to improve training efficiency. We developed a deep learning model based on mobile vision transformers and trained the model using semi-supervised learning for image classification. Compared to standard supervised learning (98.4%), our ExoNet-SSL system was able to maintain high prediction accuracy (98.8%) when tested on previously unseen environments, while requiring 35% fewer labelled images during training. These results show that semi-supervised learning can improve training efficiency by leveraging large amounts of unlabelled data and minimize the size requirements for manually annotated images. Future research will focus on model deployment for onboard real-time inference and control of human-robot walking.