Real-Time Human-Robot Interaction for a Service Robot Based on 3D Human Activity Recognition and Human-Mimicking Decision Mechanism

Abstract

In this paper, we present a real-time Human-Robot Interaction (HRI)system for a service robot based on 3D human activity recognition and human-mimicking decision mechanism. A three-layer Long-Short-Term Memory (LSTM)network is trained to recognize human activity. The input of this network is the 3D skeleton joints information captured by Kinect V1 and the output of this network is the class label of human activity. Moreover, the human-mimicking decision mechanism is also fused into the online test process of human activity recognition, which allows the robot to instinctively decide whether to interrupt the current task according to task priority. The robot can make appropriate responses based on aforementioned analytical results. The system framework is realized on the Robot Operating System (ROS). The real-life activity interaction between our service robot and the user was conducted to demonstrate the effectiveness of developed HRI system.