Addressing Sample Efficiency and Model-bias in Model-based Reinforcement Learning

Abstract

Model-based reinforcement learning promises to be an effective way to bring reinforcement learning to real-world robotic systems by offering a sample efficient learning approach compared to model-free reinforcement learning. However, model-based reinforcement learning approaches at present struggle to match the performance of model-free ones. This work attempts to fill this gap by improving the performance of model-based reinforcement learning while further improving its sample efficiency. To improve the sample efficiency, an exploration strategy is formulated which maximizes the information gain. The asymptotic performance is improved by compensating for the model-bias using a model-free critic. We have evaluated our proposed approach on four reinforcement learning benchmarking tasks in the openAI gym framework.