Object Detection and Segmentation using LiDAR-Camera Fusion for Autonomous Vehicle

Abstract

The Light detection and ranging (LiDAR) sensor plays a crucial role in perceiving the environment for an autonomous vehicle. But, in many scenarios LiDAR is unable to capture important information, for example, traffic light signals. This kind of scenario can be avoided by using camera images with LiDAR data. But, the system will not work effectively, if there is no proper calibration and synchronization between camera images and LiDAR data. In this paper, we have demonstrated a system, where objects are synchronously detected and segmented in both images and LiDAR data from KITTI datasets. Currently, the system is working in real-time using Robot Operating System (ROS) and can process up to 10 frames of image and point cloud data per second.