Sim-to-Real Transfer for AUV Fault Control with Deep Reinforcement Learning

摘要

This paper presents a methodology for the sim-to-real transfer of deep reinforcement learning (DRL) models for fault-tolerant control (FTC) in autonomous underwater vehicles (AUVs). The proposed approach leverages open-access simulation tools to train a DRL-based control reallocation strategy, enabling adaptive responses to actuator faults. Domain randomisation during DRL training is employed to enhance robustness against varying fault scenarios. The trained model is first validated in a second simulator to assess its adaptability across different simulation environments. It is then transferred to a physical AUV for real-world evaluation. Experimental results demonstrate that the DRL-based control strategy effectively maintains trajectory control despite actuator faults, outperforming standard controllers. This study highlights the feasibility of sim-to-real transfer of DRL-based FTC in underwater robotics.