A stereo-vision SLAM method based on Manhattan-plane constraints and point-plane collaborative optimization

Abstract

Abstract Previous research on visual simultaneous localization and mapping (SLAM) based on Manhattan framework constraints relied on the Manhattan world assumption, which only constrained the rotation matrix. However, although this can reduce drift in the rotation matrix, the translation matrix inevitably accumulates errors. To make full use of the structural information in the scene, this study proposes a SLAM method that utilizes two or three orthogonal plane features extracted from coplanar line features in stereo images to construct and match Manhattan plane (MP) features. By using three orthogonal MP features, the method can decouple both the absolute rotation and translation matrices simultaneously in a structure-from-motion manner, referred to as MHT-SP (MP constraint based on stereo-plane). To address the low tracking and matching rate of plane features in large-scale scenes, a method based on the joint matching of point-plane geometric constraints and coplanar line feature descriptors is proposed for plane feature tracking and matching. This method searches global and local frames without being affected by the accuracy of the camera pose in the current frame. For image frames where the MP cannot be detected, the study proposes a point-plane collaborative nonlinear optimization scheme based on a plane-feature self-attention mechanism to solve for the pose matrix. To demonstrate the effectiveness of the proposed system, this paper uses public datasets (EuRoC and KITTI) and data collected in real time by the ZED 2i camera for verification and evaluation. The results show that compared with other classic stereo camera-based ORB-SLAM3 (based on point features) Campos et al 2021 ( IEEE Trans. Robot. 37 1874–90), ORB-Line-SLAM (based on line features) Zhao et al 2022 ( Eng. Comput. 38 3847–69) and Stereo-Plane-SLAM (based on plane features) Jiang et al 2024 ( IEEE Trans. Autom. Sci. Eng. 21 1421–33) as well as RGB-D camera-based Planar-SLAM Li et al 2020 ( IEEE Robot. Autom. Lett. 5 6583–90) and Manhattan-SLAM Kim et al 2018 ( 2018 European Conf. on Computer Vision (ECCV) (Springer) pp 350–66) systems, this system achieves superior performance in both stability and accuracy.