Active Pedestrian Detection for Excavator Robots based on Multi-Sensor Fusion

Abstract

As a common multi-functional engineering equipment, excavators are widely used in civil construction, coal mining, power engineering, etc. The excellent performance of the excavator not only greatly improves the work efficiency during the construction process, but also effectively saves labor costs. However, due to the complexity of the working environment of the excavator and the blind area of the excavator itself, the driver cannot make timely judgments on the surrounding environment, which may cause potential threats to pedestrians. In response to such problems, this paper proposes a multi-sensor fusion detection method applied to excavators to provide vision assistance for excavator drivers, thereby reducing the risk of pedestrian casualties. Based on the results of the joint calibration, the transformation relationship between the camera and lidar coordinate systems is determined. Combining the detection results of the pedestrian detection algorithm YOLO-v5 and the segmented image information, the position of the pedestrian in the image can be inversely mapped to the 3D point clouds via the matrix transformation, which can accurately display the position of the pedestrian in the point clouds, consequently making up for the lack of depth information in the image. The experimental results show that our method can effectively extract the location information of pedestrians from the complex background environment and realize timely pedestrian alarm.