End-to-End Autonomous Exploration for Mobile Robots in Unknown Environments through Deep Reinforcement Learning

Abstract

Autonomous exploration in unknown environments is a significant capability for mobile robots. In this paper, we present an end-to-end autonomous exploration model based on deep reinforcement learning (DRL), which takes the sensor data and a novel exploration map as inputs, and directly outputs the motion control commands of the robot. In contrast to the existing DRL-based exploration methods, the proposed model has no requirements to be combined with the traditional exploration or navigation algorithms, resulting in lower computational complexity. We directly transfer the DRL-based model trained in the training map to four test maps with different sizes and layouts, and the results show that the robot can rapidly adapt to unknown scenes. Besides, a comparison study with RRT-exploration algorithm indicates that the proposed model can reach a higher map exploration rate within less distance and time. Furthermore, we also conduct experiments on the real physical robot to demonstrate the transferability of learned policy from simulation to reality. A video of our experiments in the Gazebo simulator and real world can be found here <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>