LiDAR-Based Autonomous Exploration Method of Mobile Robot Using Deep Reinforcement Learning in Unknown Environments

Abstract

Autonomous exploration holds significant application value in tasks like mine exploration and environmental modeling, and personnel search and rescue, effectively boosting task efficiency. Learning-based methods are well-suited for these scenarios. Nevertheless, they suffer from low learning efficiency and challenges in transferring from simulation to reality. To improve efficiency in the exploration process of mobile robots, a novel 3D LiDAR-based autonomous exploration method using deep reinforcement learning in unknown environments is proposed. In particular, a sparse informative graph based on the complementary holes is proposed, which serves as the input of our model. Furthermore, our model is built on the soft actor-critic algorithm, integrating reinforcement learning with a graph-based and self-attention mechanism policy network. Moreover, an enhanced reward in our model is also employed to boost exploration capability. The proposed method was analyzed and validated in both simulation and real-world scenarios, demonstrating superior exploration performance (11.53% in trajectory length, 13.45% in makespan, 30.77% in planning time) than the state-of-the-art (SOTA) method in a 200m×180m self-established scenario. It also demonstrates that our method enhances the efficiency and robustness of autonomous exploration tasks.