Path Planning of Unmanned Surface Vehicle Based on Improved Q-Learning Algorithm

Abstract

Efficient maritime navigation through obstructions is still one of the many problems faced by mariners. In this paper, the path planning method of mobile robots based on extensive research is used for reference, and the special requirements of Unmanned Surface Vehicle (USV) navigation process are considered. A USV path planning model based on improved Q- Learning (QL) algorithm is proposed. The improved QL algorithm is to introduce a strategy optimization selection model. The Q-Learning algorithm based on strategy optimization (SO- QL) can pre-screen behavior strategies, reduce the computational complexity of classical QL algorithm and accelerate the speed of path planning. The theoretical framework of reinforcement learning is Markov Decision Process (MDP), in which collision avoidance rules and marine environmental factors are fully taken into account. The simulation environment is built on Python and Pygame platforms. The simulation results show that the SO-QL algorithm is feasible and superior in obstacle avoidance and path planning under uncertain environmental information.