A Vision-Based Measure of Environmental Effects on Inferring Human Intention During Human Robot Interaction

Abstract

In this work, we consider the problem of sensing environmental influence on the recognition of human intention in human robot interaction (HRI). We propose a novel framework to analyze the mechanism and the characteristics of the environment in all aspects of HRI. A sensor-based deep learning method is proposed to model the conditional probability of interaction between humans and robots in different environments. Having collected RGB-D videos with 3D human skeletons and environmental features by a Kinect azure RGBD sensor, people’s intentions to interact with a robot is inferred by a discriminant sensor network, trained jointly with a LSTM and a MLP. We collect a HRI dataset with various possible environments to train models and then conduct experiments of predicting the intention of participants. Our method outperforms the traditional environment-free approach in training results (98.8% and 77.0% in testing accuracy). Experimental validations of real-time human prediction also prove the higher speed and precision of our method compared with the baseline.