Comparative Analysis of Deep Reinforcement Learning Algorithms for Hover-to-Cruise Transition Maneuvers of a Tilt-Rotor Unmanned Aerial Vehicle

摘要

Work on trajectory optimization is evolving rapidly due to the introduction of Artificial-Intelligence (AI)-based algorithms. Small UAVs are expected to execute versatile maneuvers in unknown environments. Prior studies on these UAVs have focused on conventional controller design, modeling, and performance, which have posed various challenges. However, a less explored area is the usage of reinforcement-learning algorithms for performing agile maneuvers like transition from hover to cruise. This paper introduces a unified framework for the development and optimization of a tilt-rotor tricopter UAV capable of performing Vertical Takeoff and Landing (VTOL) and efficient hover-to-cruise transitions. The UAV is equipped with a reinforcement-learning-based control system, specifically utilizing algorithms such as Deep Deterministic Policy Gradient (DDPG), Trust Region Policy Optimization (TRPO), and Proximal Policy Optimization (PPO). Through extensive simulations, the study identifies PPO as the most robust algorithm, achieving superior performance in terms of stability and convergence compared with DDPG and TRPO. The findings demonstrate the efficacy of DRL in leveraging the unique dynamics of tilt-rotor UAVs and show a significant improvement in maneuvering precision and control adaptability. This study demonstrates the potential of reinforcement-learning algorithms in advancing autonomous UAV operations by bridging the gap between dynamic modeling and intelligent control strategies, underscoring the practical benefits of DRL in aerial robotics.