Semantic mapping and semantics-boosted navigation with path creation on a mobile robot

Abstract

Movable obstacles are challenges for robotic navigation in human living environments because they lead to frequent changes in pre-built maps. Such changes cause navigation failures and high costs in path replanning. In this paper, we focus on the problem of semantic mapping and semantics-boosted navigation, introducing our intelligent robotic system that is capable of real-time belief-based scene understanding, object detection and recognition, as well as path creation strategy based on the semantic information. With deeper understanding of the environments, robotic navigation in cluttered environments filled with movable obstacles is improved. Scene understanding and object detection are achieved by our state-of-the-art Convolu-tional Neural Network (CNN) with multi-functions and then transformed into robotic beliefs using Bayes filtering to ensure temporal coherence. Object beliefs are further extended from 2D to 3D space with geometric feature detection and used for semantic mapping together with scene beliefs. Planning algorithm is designed to utilize semantic beliefs to create path through cluttered environments by manipulating movable obstacles at minimal costs. We evaluate the system on our robots with real human-living office scenarios.