Autonomous Navigation and Obstacle Avoidance using Self-Guided and Self-Regularized Actor-Critic

Abstract

This paper presents a deep reinforcement learning (DRL) algorithm for autonomous navigation and obstacle avoidance of a mobile robot. Deep Reinforcement learning algorithms (DRL) have achieved great success in sequential decision-making problems and control tasks. One of the controlling elements of deep reinforcement learning (DRL) is the target network which alleviates the divergence when learning the Q function. But, target networks slow down the learning process due to delayed learning updates. So, this makes learning unstable and induces poor performance in high-dimensional domains. As a result, it is challenging to train and deploy robots in real-time using existing Deep Reinforcement learning algorithms, which require a large number of training examples to converge or perform better in various scenarios. To tackle this issue we have used an algorithm called self-guided and self-regularized actor-critic (GRAC) which doesn't require target networks for learning state-action values in high dimensional continuous state space and continuous actions. Where we have developed an experimental test bed in the ROS-Gazebo simulator by implementing self-guided self-regularized actor-critic (GRAC) algorithm for goal-directed navigation and obstacle avoidance tasks using the Turtlebot3 robot. The simulated experiments were conducted in two different environments. In comparison to existing algorithms such as DDPG, the simulation results demonstrate that the proposed algorithm exhibits improved performance and faster convergence in navigation and obstacle avoidance of a mobile robot in high dimensional continuous state space and continuous actions.