Semantic mapping for object category and structural class

Abstract

Intelligent robots require a semantic map of the surroundings for applications such as navigation and object localization. With this information, a robot can make task planning, object manipulation and human-robot interaction. However, it still remains an open problem although considerable emphasis has been given. In this paper, we propose a novel approach to generate a dense semantic map for 3D indoor scene. Our approach integrates a robust image labeling algorithm with simultaneous localization and mapping method (SLAM) to generate the semantic map. Scene information, semantic context and geometric context are encoded into a CRF model. Our CRF model computes a simultaneous labeling of image regions into semantic classes (e.g., bed, table, chair) and structural object classes (Ground, Furniture, Structure, Props). Then semantic labeling results in single images are fused into the 3D map using the estimated camera poses by SLAM. We report our labeling performance on NYU v2 dataset and demonstrate that our algorithm is comparable to and in many cases superior to the previous method. Also we generate our semantic map on the NYU v2 video dataset.