Fusing Panoptic Segmentation and Geometry Information for Robust Visual SLAM in Dynamic Environments

Abstract

Mobile robots need reliable maps for autonomous operation. Traditional SLAM systems, which are mainly developed for static scenes, often fail in dynamic environments with moving objects present in the scene. Learning based dynamic SLAM systems often suffer from insufficient or inaccurate identification of feature points. This paper proposes a novel real-time RGB-D SLAM system, which is targeted for dynamic environments, can further enhance feature detection and dynamic removal. This is done by fusing panoptic segmentation and geometry information. The system includes four components: dynamic segmentation that reduces the impact of moving objects, pose estimation with dynamic object removal, panoptic mapping, and loop closing. The pose estimation uses coarse-to-fine dynamic/static classification to further reduce the impact of unknown moving objects. Extensive evaluations demonstrate that our system can achieve robust performance in complex dynamic environments.