Intelligent path planning of mobile robot based on Deep Deterministic Policy Gradient

Abstract

Abstract Deep Deterministic Policy Gradient (DDPG) is a deep reinforcement learning algorithm that is widely used in the path planning of mobile robots. It solves the continuous action space problem and can ensure the continuity of mobile robot motion using the Actor-Critic framework, which has great potential in the field of mobile robot path planning. However, because the Critic network always selects the maximum Q value to evaluate the actions of mobile robot, there is the problem of inaccurate Q value estimation. In addition, DDPG adopts a random uniform sampling method, which can’t efficiently use the more important sample data, resulting in slow convergence speed during the training of the path planning model and easily falling into local optimum. In this paper, a dueling network is introduced based on DDPG to improve the estimation accuracy of the Q value, and the reward function is optimized to increase the immediate reward, to direct the mobile robot to move faster toward the target point. To further improve the efficiency of experience replay, a single experience pool is separated into two by comprehensively considering the influence of average reward and TD-error on the importance of samples, and a dynamic adaptive sampling mechanism is adopted to sample the two experience pools separately. Finally, experiments were carried out in the simulation environment created with the ROS system and the Gazebo platform. The results of the experiments show that the proposed path planning algorithm has a fast convergence speed and high stability, and the success rate can reach 100% and 93% in the environment without obstacles and with obstacles, respectively.