Cooperative Deep Reinforcement Learning Policies for Autonomous Navigation in Complex Environments

Abstract

A critical part of achieving robust and safe navigation for mobile robots is selecting the right navigation policies trained through simulation to operate effectively in real-world situations. Simulation-trained policies often struggle for mobile robot settings deployed in real-world navigation tasks, leading to policy degradation and increased risk manners. To address these challenges, a cooperative deep reinforcement learning policies (CDRL) framework is proposed, ensuring safe exploration and deployment in unknown complex environments. The CDRL framework cooperates with exploration and exploitation policies based on a policy-switching mechanism, which efficiently helps the robot escape the local optima. Instead of transferring a single navigation policy, CDRL leverages cooperative navigation policies with diverse reward functions, enabling them to adapt to unknown complex environments. The proposed technique is based on an exploration distributional soft actor critic (E-DSAC) and soft actor critic (SAC) algorithms, which enhances training efficiency. The deep reinforcement learning (deep RL) models in this framework are represented by a mobile service robot that reaches target positions without requiring a map presentation. Experimental results show that the proposed framework is proven to have safe and fast motions in terms of navigation time and success rates. The sim-to-real transfer process of mobile service robots can be found ( <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://youtu.be/vIxRqXidKIM</uri> ).