Pedestrian detection and tracking based on 2D Lidar and RGB-D camera

Abstract

Aiming at the requirement of the mobile robot for the human-robot coexisting environment to distinguish people from objects and accurately estimate pedestrian pose and speed information in the indoor environment, the pedestrian detection and tracking method based on 2D Lidar and RGB-D camera is proposed. Firstly, 2D Lidar and environmental prior information are used to detect the position of leg features of the pedestrian. Then the RGB-D camera is used to detect the human skeleton features to obtain the pose of the pedestrian legs. Finally, based on the pedestrian constant velocity motion model and the observation model of leg feature, Kalman filter and global nearest neighbor data association are used to realize pedestrian motion state tracking. Experiments on the dataset demonstrate that the use of prior maps can reduce 2D Lidar false detections by 54.17% and improve the maximum and average persistent tracking time by 3.67 and 1.18 times. In the real scene experiment, the static pedestrian pose detection and the accurate tracking of the dynamic pedestrian are realized, which solves the problem that 2D Lidar cannot recognize the pose of humans when they are stationary.