Efficient Reinforcement Learning for 3D LiDAR Navigation of Mobile Robot

Abstract

Developing an efficient automatic navigation system for mobile robots is challenging in the strange scenarios where robots can only observe the environment of the surrounding limited area. While other distributed automatic navigation systems exist, they often require extracting semantic information to calculate navigation action, which requires extra modules to provide perceptual information and is not robust. We propose an end-to-end automatic navigation system based on the reinforcement learning technology. In particular, the raw 3D LiDAR data is used to directly map an efficient navigation policy. We design a novel dense reward function to handle the reward sparsity issue and provide a graphical representation method to enable the efficient feature learning from the raw 3D LiDAR data in our navigation system. In addition, an imitation learning based policy initialization is introduced before the subsequent reinforcement learning, which increases the learning efficiency and, in the meantime, still encouraging the robot to explore all the potential states to achieve advanced performance than the imitated experts. Our navigation model is trained in the Webots environment and the experimental results show that our model has efficient and flexible navigation performance in complex environments. More importantly, trained model can be easily extended to unfamiliar environments.