Synthesized semantic views for mobile robot localization

摘要

Localizing a mobile robot in a given map is a crucial task for autonomy. We present an approach to localize a robot equipped with a camera in a known 2D or 3D geometrical map that is augmented with semantic information (e.g., a floor plan with semantic labels). The approach uses semantic information to mediate between the visual information from the camera and the geometrical information in the map. Moreover, semantic information is robust to appearance changes like lighting conditions. Instead of solely relying on salient semantic landmarks (i.e., "things" like doors) we also exploit "stuff"-like semantic classes such as wall and floor. The presented localization approach builds upon the idea of computing a semantic segmentation of an incoming camera image using a Convolutional Neural Network and subsequent matching to semantic views synthesized from a map. We give details about the algorithmic approach on how to semantically segment images, synthesize images from the semantic 2D or 3D map, the matching between images from both sources, and the integration in Monte Carlo localization. Further, we provide a set of proof-of-concept experiments and evaluate the influence of the selected set of semantic classes. To work towards the usage of hand-drawn sketches as input map, we also evaluate the robustness of the presented approach to map distortions.