Learning optimal navigation actions for foresighted robot behavior during assistance tasks

Abstract

We present an approach to learn optimal navigation actions for assistance tasks in which the robot aims at efficiently reaching the final navigation goal of a human where service has to be provided. Always following the human at a close distance might hereby result in inefficient trajectories, since people regularly do not move on the shortest path to their destination (e.g., they move to grab the phone or make a note). Therefore, a service robot should infer the human's intended navigation goal and compute its own motion based on that prediction. We developed an approach that applies reinforcement learning to get a Q-function that determines for each pair of the robot's and human's relative positions the best navigation action for the robot. Our approach applies a prediction of the human's motion based on a softened Markov decision process (MDP). This MDP is independent from the navigation learning framework and is learned beforehand on previously observed trajectories. We thoroughly evaluated our method in simulation and on a real robot. As the experimental results show, our approach leads to foresighted navigation behavior and significantly reduces the path length and completion time compared to naive following strategies.