Multi-View Human Pose Estimation in Human-Robot Interaction

Abstract

This paper contributes a real-time human-robot interaction system based on multi-human poses capture. First, we propose an iterative method for multi-human 3D poses estimation from multi-view. There are three mainly steps: (1) all independent 2D poses under each view are obtained by using the pose detector; (2) 2D poses from different perspectives are associated based on a greedy algorithm; (3) 3D skeletons are generated with the 2D poses that have been correctly associated. In the process of each iteration, the poses with the highest correspondence are always paired priority to ensure the accuracy of the results. Due to the occlusion in each view, previous methods that rely on the appearance of the human body and wearable devices are not suitable for multi-human 3D poses estimation, especially in crowded scenes. The proposed method does not required any markers or devices, which has good robustness even under occlusion or crowded. In addition, we also designed three human-robot interaction modes, including action following, target specification and dynamic obstacle avoidance. For each mode, target generation and correction methods are present. Combining human body pose estimation and robot motion, design experiments to verify the smoothness of the robot control method for dynamic target tracking and the effectiveness in different interaction modes.