Real-Time Person-Following Robot: Front-Following Using Human Motion Prediction

Abstract

Many studies have been conducted on companion robots that follow behind a human leader; however, this strategy puts the robot out of sight of the person it is accompanying. To stay within sight, the robot needs to follow the leader from a different position. This paper presents a front-following system for an autonomous mobile robot using a Kinect sensor. Research effort is concentrated on control of the robot, which walks in front of the human leader. For a general human-following system, especially for front-following, both localization of the robot and the prediction of the human's motion and state position are necessary. However, the framework proposed in this study uses a machine-learning-based prediction system to direct the robot ahead of the human without the need for robot localization. The proposed human motion prediction neural network predicts the 3D coordinates of a human walking behind the robot and, when combined with a proportional-integral-differential controller to control robot movement, enables accurate following for turning angles up to <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$100^{\circ}$</tex>. Since robot localization is not required, only one Kinect sensor is needed. The front-following system is validated via both simulation and real-time experiments, demonstrating overall success in front-following for wide and narrow spaces.