PMOD-Net: point-cloud-map-based metric scale obstacle detection by using a monocular camera

Abstract

Metric scale obstacle detection, which detects obstacles and measures the distances to them with a metric scale, is a key function in autonomous driving. A monocular camera is inexpensive and effective for detecting objects in images. However, it cannot measure the distances to objects with a metric scale because it can estimate only relative distance. 3D point cloud maps can determine the distances of fixed objects in the 3D map; however, they cannot detect non-fixed obstacles that are not in the 3D map. Therefore, we developed a new method for detecting non-fixed obstacles using a monocular camera and 3D point cloud maps. We used a semantic segmentation neural network (NN) for detecting obstacles and an image-guided depth completion NN for densifying a sparse depth map with a metric scale. We proposed a multitask NN that three-dimensionally reconstructed non-fixed obstacles using the shape information obtained by the semantic segmentation NN. The detection accuracy of the proposed multitask NN was 1.3 times higher than that of a single-task method. Moreover, our robot avoided obstacles using the proposed NN.