Semantic Map Augmentation for Robot Navigation: A Learning Approach Based on Visual and Depth Data

Abstract

In this paper, we propose a framework to build an improved metric representation of the environment with semantic information. We explore some of the recent advances of deep neural networks to object detection/semantic classification in a visual-based perception scheme. The output of this system is a map of the environment extended with semantic object classes and their positioning. This framework combines sensors available in commonly used mobile robotic platforms such as an RGB-D camera, 2D LIDAR and odometers. In short, a CNN-based object detector and a 3D model-based segmentation technique are used to localize and identify different classes of objects in the scene. Then the tracking of the semantic classes is performed with a Kalman filter approach. We show results for "door" objects and validate this approach with a collected dataset in an extensive indoor area, comprising corridors and offices. A dataset and the source code are made available to the community as ROS packages.