Autonomous Robot Navigation System Without Grid Maps Based on Double Deep Q-Network and RTK-GNSS Localization in Outdoor Environments

Abstract

This paper proposes an autonomous mobile robot navigation system without grid maps in outdoor environments. The system integrates local navigation based on deep reinforcement learning and localization using RTK-GNSS. Local navigation is to travel between waypoints by using a learned policy. Localization is required for state of learning-based navigation and arrival evaluation of waypoints. First, a robot learns a policy traveling between waypoints in an environment that imitates an actual outdoor environment and avoiding collision with obstacles in our original simulator. DDQN(Double Deep Q-Network) is applied as an learning algorithm. We aim for learning that a robot can take an adequate action from obstacle positions obtained from 2D-LiDAR, a relative distance and a relative angle to a destination. Then, a robot performs navigation in outdoor environments based on the learned policy. Experimental results include learning in several environments, accuracy of RTK-GNSS and the integrated navigation system in an actual outdoor environment. Especially, our proposed system could travel approximately 600[m] in a general urban environments.