Model-Based Reinforcement Learning For Robot Control

Abstract

Model-free deep reinforcement learning (MFRL) algorithms have achieved many impressive results. But they are generally stricken with high sample complexity, which puts forward a critical challenge for their application to real-world robots. Dynamic models are essential for robot control laws, but it is often hard to obtain accurate analytical dynamic models. Therefore a data-driven approach to learning models becomes significant for reinforcement learning to increase data efficiency. Model-based algorithms are effective methods to reduce sample complexity by learning the system dynamic model. However, in certain environments, it has been proven that learning an accurate system dynamic model is a formidable problem, and their asymptotic performance cannot achieve to the same level as model-free algorithms. In our work, we use an ensemble of deep neural networks to learn system dynamics and incorporate model uncertainty. Then in order to merge the high asymptotic performance of the advanced model-free methods, the deep deterministic policy gradient (DDPG) algorithm is adopted to optimize robot control policy. Furthermore, it has been implemented within ROS for controlling a Baxter robot in the simulation environment.