Modelling inter-task relations to transfer robot skills with three-way RBMs

摘要

Transfer on Reinforcement Learning (RL) is a promising method on learning new skills and adapting new situations for humanoid robots as tasks or environments change. Within the normal process of Transfer Learning in RL, the inter-task mapping is manually defined, which lacks generalization ability. Therefore, how to automatically learning intertask relations becomes a hot topic. Considering the limited computational resource of a physical humanoid robot, high learning efficiency regarding to both fast speed algorithm and low sample complexity should be emphasized in skills transfer. According to this view, in this research, the inter-task relations are modelled using a three-way Restricted Boltzmann Machine (RBM), which is turned out to be a powerful model in capturing the similarity between samples from source task and target task. Since standard Contrastive Divergence (CD) algorithm commonly used for RBM learning suffers from the inputindependent problem and may lead the learning process timeconsuming or inapplicable, a Cyclic Contrastive Divergence (CCD) learning algorithm is employed. In order to evaluate the performance, experiment that transfer the skill of walking on flat surface to the skill walking on slope surface is conducted on our physical robot platform, PKU-HR5.1, and the result indicates that the method is feasible and efficient.