Evolving Childhood's Length and Learning Parameters in an Intrinsically Motivated Reinforcement Learning Robot

摘要

The capacity of re-using previously acquired skills can greatly enhance robots ’ learning speed and behavioral complexity. ‘Intrinsically Motivated Reinforcement Learning (IMRL)’ is a framework that exploits this idea and proposes to build agents capable of solving several specific tasks by assembling general-purpose building-block behaviors (‘skills’) previously acquired on the basis of ‘intrinsic motivations’. This paper proposes a novel neural-network hierarchical reinforcement-learning architecture which exploits ‘evolutionary robotics (ER) ’ techniques that not only allow tackling important limits of IMRL, as shown in previous papers, but they also allow investigating two other important issues, namely: (1) the optimization of the parameters that regulate the architecture’s learning processes; (2) the optimization of the time the architecture dedicates to the acquisition of the skills ’ repertoire. These two issues are investigated here through a simulated robot engaged in solving compositional path-following navigation tasks. The main results obtained indicate that the proposed approach allows obtaining a remarkable improvement of performance of the architecture, while at the same time decreasing the time the system needs to learn the skills (‘childhood’), with respect to cases where hand-tuned parameters are used.