Indoors Traversability Estimation with RGB-Laser Fusion

Abstract

We propose a dual-stream, semi-supervised, attention-based approach that employs feature fusion of RGB and Laser Range Finder (LRF) modalities. Our method lever-ages the strength of two powerful transformer-based networks, i.e. Vision Transformer (ViT) and SegFormer, along with LRF information, to adequately predict whether the scene encountered in the image is safe for a robot to traverse. Towards this effort, we introduce an automated labelling system profiting from the combination of raw velocity readings and laser scanning information. Moreover, we show that overall GOINO-GO detection is enhanced by fusing RGB and laser modalities. Feature fusion is achieved through the employment of a Multi-Head Self-Attention (MHSA) module. Through cross-domain validation, we show that the proposed traversability estimation method can achieve decent amounts of transferability even with limited amount of training data.