An Improvement on Mapless Navigation with Deep Reinforcement Learning: A Reward Shaping Approach

Abstract

This paper presents an algorithm for mapless motion planning with deep reinforcement learning. Focusing mainly on reward shaping, possible reward functions for the navigation problem are investigated with the deep deterministic policy gradient method. Accordingly, a new reward function to improve the robot's performance is proposed. The input vector consists of 20 distance laser data, the relative goal position, and the last step velocity, while the output is the velocity command. The proposed method improves previous similar algorithms and shows an acceptable performance in a dynamic environment. The results reveal that the new reward function's effect on the algorithm generates smoother moves of the robot, which results in a 40 percent time reduction and a 30 percent total error reduction compared to the original method. Besides, the learning is completed much faster and takes only 5 hours, while the original method takes 16 hours. All the training and testing processes are conducted on a standard personal laptop.