Robot Mapless Navigation in VUCA Environments via Deep Reinforcement Learning

Abstract

Mobile robots operating in unknown social environments demand the ability to navigate among crowds and other obstacles in a safe and socially compliant manner without prior maps. This work proposes a deep reinforcement learning framework for robot mapless navigation in such unknown congested and cluttered scenarios. A value network integrating crowd and static obstacle information is developed for robot decision-making, where spatial-temporal reasoning and lidar map are leveraged to comprehend the surrounding environment. Based on the relative velocities between the robot and humans, the hazardous areas that the robot should avoid are formulated. Accordingly, an original reward function is put forward for safe and socially compliant robot navigation. Extensive simulation experiments demonstrate the superiority of the proposed framework, which outperforms the state-of-the-art methods in terms of success rate (up to 44% increase) and discomfort frequency (up to 74.28% decrease). Additionally, we validate the real-time performance and practicality of our approach by successfully navigating a robot in real-world complicated scenes.