Comparing direct and indirect encodings using both raw and hand-designed features in tetris

Abstract

Intelligent agents have a wide range of applications in robotics, video games, and computer simulations. However, fully general agents should function with as little human guidance as possible. Specifically, agents should learn from large collections of raw state variables instead of small collections of hand-designed features. Learning from raw state variables is difficult, but can be easier when agents are aware of the geometry of the input space. Indirect encodings allow agents to take advantage of the geometry of the task, and scale up to large input spaces. This research demonstrates the relative benefits of a direct and indirect encoding using raw or hand-designed features in Tetris, a challenging video game. Specifically, the direct encoding NEAT is compared against the indirect encoding HyperNEAT Both algorithms create neural networks to play the game, but HyperNEAT makes better use of raw screen inputs, due to its ability to generate large networks that take advantage of the domain's geometry. However, hand-designed features lead to higher scores with both algorithms. HyperNEAT makes better use of hand-designed features early in evolution, but NEAT eventually overtakes it. Since each method succeeds in different circumstances, approaches combining the strengths of both should be explored.