Improved M-DQN With $\epsilon$-UCB Action Selection Policy and Multi-Goal Fusion Reward Function for Mobile Robot Path Planning

Abstract

Both DQN and M-DQN is commonly used for mobile robot path planning, with the problem of relatively low efficiencies of the action selection policy and the reward function when coping with complex environments. To solve the problem, an improved <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\epsilon$</tex-math></inline-formula>-UCB action selection method based on neural network is proposed to fit the uncertainty of state-action pairs, which effectively improves the exploration efficiency. Meanwhile, aiming at the effectiveness of the reward function, a multi-goal fusion reward function is proposed to improve the guidance ability. Further, the proposed <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\epsilon$</tex-math></inline-formula>-UCB action selection policy and the multi-goal fusion reward function are introduced into the M-DQN framework to complete the path planning task of the mobile robot. Finally, some comparative experiments are carried out to verify the effectiveness of the proposed method. The experimental results show that the proposed method exhibits faster convergence speed and higher learning efficiency for the path planning of mobile robots in complex environments than other existing methods.