Deep Reinforcement Learning Based Three-dimensional Path Tracking Control of An Underwater Robot

Abstract

Abstract This paper presents a deep reinforcement learning (DRL)-based three-dimensional path tracking control algorithm of an underwater robot to learn the path-tracking capability by interacting with the environment. To be specific, a hybrid path tracking guidance and controller based on three-dimensional line-of-sight (3D LOS) guidance and twin delayed deep deterministic policy gradient (TD3) algorithm is applied to complete kinematics and dynamics controller design. The reference angle is obtained by LOS algorithm, and TD3 algorithm is used to output the control laws. Aiming at the chattering problem in the output of the reinforcement learning controller, the commands filter and chattering penalty term are designed respectively. The tracking experiment of ten waypoints proves the feasibility of the algorithm proposed in this paper.