A dense semantic mapping system based on CRF-RNN network

Abstract

Geometric structure and appearance information of environments are main outputs of Visual Simultaneous Localization and Mapping (Visual SLAM) systems. They serve as the fundamental knowledge for robotic applications in unknown environments. Nowadays, more and more robotic applications require semantic information in visual maps to achieve better performance. However, most of the current Visual SLAM systems are not equipped with the semantic annotation capability. In order to address this problem, we develop a novel system to build 3-D Visual maps annotated with semantic information in this paper. We employ the CRF-RNN algorithm for semantic segmentation, and integrate the semantic algorithm with ORB-SLAM to achieve the semantic mapping. In order to get real-scale 3-D visual maps, we use the RGB-D data as the input of our system. We test our semantic mapping system with our self-generated RGB-D dataset. The experimental results demonstrate that our system is able to reliably annotate the semantic information in the resulting 3-D point-cloud maps.