Develop A Navigation Approach for Mobile Robots Based on the Distributional Deep Reinforcement Learning Framework

Abstract

With traditional reinforcement learning, mobile robots can learn the expected value of each action in every state. However, in environments with uncertain situations, such as the sudden appearance of obstacles, the expected value may not accurately reflect the situation, leading to unsuccessful navigation for the robot. Distributional Deep Reinforcement Learning can help the mobile robot learn the distribution of future outcomes of actions. Instead of predicting a single expected value, the model will predict a distribution of positions that may be reached after each action. In this paper, a novel Quantile Regression Deep Q-Network (QR-DQN) algorithm is proposed to address the obstacle avoidance problem on a mobile robot based on depth images. Moreover, the designed model concentrates on integrating depth information and its application in the control system via the neuron-network sequence layers. The simulation results reveal that the proposed method outperforms the other Deep Reinforcement Learning method under different scenarios.