Meta Reinforcement Learning with Generative Adversarial Reward from Expert Knowledge

Abstract

Meta learning has been widely applied in the field of multi-task Reinforcement Learning. In meta-learning, a meta-model is obtained through a large number of pre-trainings and is able to adapt quickly and well on the unseen task in test. However, previous meta Reinforcement Learning methods often require vast computation cost and well-designed reward function, which are hardly available in many real world tasks, such as self-driving and robots control. On the other hand, in such cases there exist plenty of demonstrations from experts. In order to alleviate the above problems, this paper proposes a meta learning method with expert knowledge in sparse reward scenario. By introducing expert knowledge into a meta learning framework, the training speed and generalization performance of a meta-model are enhanced. Experiments show that our method can effectively improve the training speed of the meta-model. In addition, in sparse reward setting, the convergence speed, as well as the generalization ability of the proposed method, are significantly better than classic meta learning method.