RelaI2P: Relational Learning for Image-to-Point Cloud Registration

Abstract

Cross-modality registration between 2D images and 3D point clouds is an important task in autonomous driving and robotics. Existing methods predict the correspondence between images and point clouds by matching patterns of pixel and point features learned by deep neural networks. However, due to the significant differences in their representation and feature processing, the feature spaces are vastly different. The insufficient feature interaction between image and point cloud branches leads to a lack of information necessary for relational reasoning, which is crucial for establishing the correspondence between pixels and points. To address these problems, we propose a Cross-Modality Relation Module (CMRM) that leverages the relations between different levels of features from images and point clouds. This module facilitates rich information exchange between the feature extractor branches corresponding to the two modalities. Additionally, we introduce a Relation-Aware Fusion Module (RAFM) that effectively integrates multimodal features and their relations. The experimental results on KITTI dataset show improvements over the state-of-the-art methods. The code will be publicly available at https://github.com/JLUrob/RelaI2P.