Self-Motivated Development Through Rewards for Predictor Errors / Improvements

Abstract

Even in absence of external reward, babies and scientists and others explore their world.Using some sort of adaptive predictive world model, they improve their ability to answer questions such as: what happens if I do this or that?They lose interest in both the predictable things and those predicted to remain unpredictable despite some effort.We can design curious, self-motivated robots that do the same.The author's old basic principles for doing so: a reinforcement learning (RL) controller is rewarded whenever its action sequences result in predictor errors (1990), or, more generally, predictor improvements (1991).We briefly review the history of these ideas.