Learning high-level robotic soccer strategies from scratch through reinforcement learning

Abstract

The field of automated learning has been steadily growing in robotic tasks. This phenomenon is supported by the evolution of computational resources and new reinforcement learning algorithms. Researchers have drawn their attentions to methods that are easy to implement and tune, while achieving state-of-the-art performance. This trend also affects the world of robotic soccer, where new papers delve systematically into the optimization of basic skills. However, when learning higher-level strategies, there is space for improvement on two fronts. First, the simulation environment should allow the agent to abstract from low-level details. Second, the existing methods to train this kind of behaviors are still scarce. This paper contributes with innovative problem-solving methods, specifically in the rewards field. To test alternative approaches, an extended version of the RoboCup's official Soccer Server simulator was used. The results have confirmed the importance of the proposed reward components and their relationship with the episodes' initial conditions.