Efficient reinforcement learning: model-based Acrobot control

Abstract

Several methods have been proposed in the reinforcement learning literature for learning optimal policies for sequential decision tasks. Q-learning is a model-free algorithm that has previously been applied to the Acrobot, a two-link arm with a single actuator at the elbow that learns to swing its free endpoint above a target height. However, applying Q-learning to a real Acrobot may be impractical due to the large number of required movements of the real robot as the controller learns. This paper explores the planning speed and data efficiency of explicitly learning models, as well as using heuristic knowledge to aid the search for solutions and reduce the amount of data required from the real robot.