Modulation Based Transfer Learning of Motivational Cues in Developmental Robotics

Abstract

The modeling of utility is an important problem in many fields, including reinforcement learning. However, when considering a developmental approach to open-ended learning a new aspect arises. In these settings, the efficiency of the modeling process becomes a key aspect, as these processes usually take place in real time and, to increase survivability, it is necessary for the robot to be able to produce utility models as fast as possible. In this paper, we address this issue by proposing a modulation-based approach to the adaptation of the robot's experience, in the form of previously obtained ANN based utility models, to new situations. These previous utility models are perceptually recalled from a Long-Term Memory and combined to produce an initial guess to the new utility model. After this, modulatory structures are created that lead to the fine adaptation of these initial guesses to the real utility model of the new situation. Some initial results of experiments using a real robot are presented to clarify the approach. Specifically, three realistic problems that a Baxter "cooking robot" must solve are faced with this modulating approach. With them, it is clearly shown the increase in efficiency of the utility model learning in real time.