Bayesian Optimization with Automatic Prior Selection for Data-Efficient\n Direct Policy Search

Abstract

One of the most interesting features of Bayesian optimization for direct\npolicy search is that it can leverage priors (e.g., from simulation or from\nprevious tasks) to accelerate learning on a robot. In this paper, we are\ninterested in situations for which several priors exist but we do not know in\nadvance which one fits best the current situation. We tackle this problem by\nintroducing a novel acquisition function, called Most Likely Expected\nImprovement (MLEI), that combines the likelihood of the priors and the expected\nimprovement. We evaluate this new acquisition function on a transfer learning\ntask for a 5-DOF planar arm and on a possibly damaged, 6-legged robot that has\nto learn to walk on flat ground and on stairs, with priors corresponding to\ndifferent stairs and different kinds of damages. Our results show that MLEI\neffectively identifies and exploits the priors, even when there is no obvious\nmatch between the current situations and the priors.\n