Learning Autonomous Marine Behaviors in MOOS-IvP

Abstract

Manually authoring and testing behaviors for autonomous marine vehicles can become tedious and impractical when faced with complex or rapidly changing adversarial situations. We address this problem by learning autonomous behaviors using deep reinforcement learning. We apply deep reinforcement learning, an approach that learns behaviors without relying on explicit vehicle models, to a game of capture the flag with multiple competing vehicles. We integrated deep reinforcement learning with MOOS-IvP, a software suite for marine robotics communication, control, and simulation, that allows the development and execution of behaviors for both underwater and surface vehicles. To our knowledge, this is the first application of reinforcement learning to this platform. We extended MOOS-IvP to create and train a neural net to learn autonomous behaviors for reaching the opponent's flag while avoiding an adversary exhibiting a defense behavior in simulation.