Learning Roles: Behavioral Diversity in Robot Teams

Abstract

This paper describes research investigating behavioral specialization in
\nlearning robot teams. Each agent is provided a common set of skills (motor
\nschema-based behavioral assemblages) from which it builds a task-achieving
\nstrategy using reinforcement learning. The agents learn individually to
\nactivate particular behavioral assemblages given their current situation and
\na reward signal. The experiments, conducted in robot soccer simulations,
\nevaluate the agents in terms of performance, policy convergence, and
\nbehavioral diversity. The results show that in many cases, robots will 
\nautomatically diversify by choosing heterogeneous behaviors. The degree of 
\ndiversification and the performance of the team depend on the reward 
\nstructure. When the entire team is jointly rewarded or penalized (global 
\nreinforcement), teams tend towards heterogeneous behavior. When agents 
\nare provided feedback individually (local reinforcement), they converge 
\nto identical policies.