Mapless Navigation for Mobile Robots Based on Improved Soft Actor-Critic Algorithm

Abstract

Aiming at the problem of low training efficiency and slow convergence in deep reinforcement learning algorithms for mapless navigation tasks, this paper proposes an Improved Soft Actor-Critic (ISAC) algorithm based on the original Soft Actor-Critic (SAC) algorithm. The ISAC introduces an advantage structure into the critic network to improve the speed of policy learning. A reward function that includes action information and minimum obstacle distance information is designed to improve training efficiency. Through simulation experiments, the ISAC algorithm is shown to improve the convergence speed of the model during training. In policy model evaluation, the policies trained by the ISAC algorithm have better navigation performance and generalization ability than those trained by the original SAC algorithm in the same number of training episodes. In addition, we validated the effectiveness of the models in a real environment.