Optimizing High-dimensional Learner with Low-Dimension Action Features

Abstract

Model-free reinforcement learning is capable of learning high-dimensional robotic tasks, but the requirement of large-scale training data makes it hard to reach better performance in limited time. On the contrary, model-based methods are capable of learning low-dimensional tasks efficiently, but lack of extensibility for complex robotic tasks. It is an instinct that, combining the advantage of both, transferring knowledge to higher dimension may benefit in sample efficiency and model accuracy. In the thesis, we present a hybrid framework that transfer low-dimensional action features to high-dimensional deep reinforcement learning model through imitation learning, in order to decrease the training data needed to reach practical performance. In this work, the hybrid framework is experimented on the simulated locomotion tasks, showing that our framework can improve model-free learning process. Our hybrid algorithm outperforms the pure model-free method, utilizing the low-dimensional action features efficiently and being competent in model accuracy.