Curiosity Driven Reinforcement Learning for Motion Planning on Humanoids

Abstract

Most previous work on textit{artificial curiosity} and textit{intrinsic motivation} focuses on basic concepts and theory. Experimental results are generally limited to toy scenarios, such as navigation in a simulated maze, or control of a simple mechanical system with one or two degrees of freedom. To study artificial curiosity in a more realistic setting, we emph{embody} a curious agent in the complex iCub humanoid robot. Our novel reinforcement learning framework consists of a state-of-the-art, low-level, reactive control layer, which controls the iCub while respecting constraints, and a high-level curious agent, which explores the iCub's state-action space through information gain maximization, learning a world model from experience, controlling the actual iCub hardware in real-time. To the best of our knowledge, this is the first ever embodied, curious agent for real-time motion planning on a humanoid. We demonstrate that it can learn compact Markov models to represent large regions of the iCub's configuration space, and that the iCub explores textit{intelligently}, showing textit{interest} in its physical constraints as well as in objects it finds in its environment