Hierarchical controller learning in a First-Person Shooter

摘要

We describe the architecture of a hierarchical learning-based controller for bots in the First-Person Shooter (FPS) game <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Unreal Tournament 2004</i> . The controller is inspired by the subsumption architecture commonly used in behaviour-based robotics. A behaviour selector decides which of three sub-controllers gets to control the bot at each time step. Each controller is implemented as a recurrent neural network, and trained with artificial evolution to perform respectively combat, exploration and path following. The behaviour selector is trained with a multiobjective evolutionary algorithm to achieve an effective balancing of the lower-level behaviours. We argue that FPS games provide good environments for studying the learning of complex behaviours, and that the methods proposed here can help developing interesting opponents for games.