Boosting Deep Reinforcement Learning-Based Path Planning for Robotic Manipulators With Egocentric State Space Descriptions

Abstract

In robotic path planning tasks, Reinforcement Learning agents typically receive global or relative Euclidean coordinates, e.g., with respect to a target reference point as direct state information. Nevertheless, a more egocentric view of the environment seems to be favorable – based on information in polar or spherical coordinates about objects surrounding the robot. Using the model-free, actor-critic algorithm Twin Delayed Deep Deterministic Policy Gradient in combination with Prioritized Experience Replay, the advantages of an alternative definition of states using egocentric TCP-coordinates is evaluated and compared in simulations to classical approaches within two typical environments. The training results indicate a tremendous potential of the egocentric state space definition that not only offers faster learning but also more successful trainings.