Calibration-Net:LiDAR and Camera Auto-Calibration using Cost Volume and Convolutional Neural Network

Abstract

A fusion of multi-sensor has been utilized widely for improving the environment perception in autonomous vehicles and robot navigation. Calibration is an essential procedure for preprocessing the data fusion between multiple sensors. Most target-based calibration techniques require manual works and specific calibration targets to achieve high accuracy. It gradually becomes outmoded for Light Detection and Ranging (LiDAR) and camera with the development of deep learning techniques. This paper proposed an online LiDAR-camera calibration that automatically predicts the extrinsic parameters by taking advantage of convolutional neural networks (CNNs). We take depth maps of stereo camera prediction and depth maps of the LiDAR projection as two separated branches as inputs for the proposed network. Unlike the current CNN-based calibration method, we construct a cost volume of the correlation between two corresponding pixels of depth maps in stereo camera and LiDAR, respectively. The proposed model gains a reasonable capability to adjust to different initial calibration error ranges. We evaluate the proposed architecture on the KITTI dataset and achieve 0.378 degree in rotation error and 2.353cm translation error.