Developmental robotics, optimal artificial curiosity, creativity, music, and the fine arts

摘要

Even in the absence of external reward, babies and scientists and others explore their world.Using some sort of adaptive predictive world model, they improve their ability to answer questions such as what happens if I do this or that?They lose interest in both the predictable things and those predicted to remain unpredictable despite some effort.One can design curious robots that do the same.The author's basic idea (1990, 1991) for doing so is a reinforcement learning (RL) controller is rewarded for action sequences that improve the predictor.Here, this idea is revisited in the context of recent results on optimal predictors and optimal RL machines.Several new variants of the basic principle are proposed.Finally, it is pointed out how the fine arts can be formally understood as a consequence of the principle: given some subjective observer, great works of art and music yield observation histories exhibiting more novel, previously unknown compressibility/regularity/predictability (with respect to the observer's particular learning algorithm) than lesser works, thus deepening the observer's understanding of the world and what is possible in it.