Object Detection and Navigation of a Mobile Robot by Fusing Laser and Camera Information

Abstract

While state-of-the-art YOLO approaches have revolutionized real time object detection in mobile robotics, most of the publicly available models are trained on datasets with a small number of available classes. In addition, the difficulty in creating large datasets with many available classes for 2D object detection sets limitations to real world robotic applications and specialized use cases. This paper presents a solution that tackles these limitations by approaching object detection via fusion of 2D laser and RGB camera information resulting to a detector with 1000 learned classes. Object localization is performed in the 3D world by clustering the point cloud provided by the 2D laser scanner using the DBSCAN algorithm. The clusters are projected onto the image plane providing Regions of Interest (ROI), where proposed object bounding boxes are obtained, that are labeled with distance information. Object recognition is achieved using a pretrained, on the ImageNet dataset, ResNet and a voting schema among proposed bounding boxes, that also estimates the objects height. The detection system is used in combination with a navigation system that employs artificial potential field. The combination of the two, makes the robot’s perception easily adaptable to specialized applications and the robot’s behaviour adjustable to the complexity and variability of unstructured and unknown workspaces. The method has been implemented in ROS and tested both in simulation as well as in real case scenarios using the mobile robot Pioneer 3-DX. The work is aimed at robots with limited hardware and sensor capabilities and tries to enable detection via fusion, despite the limitations.