Model-Based Reinforcement Learning for Trajectory Tracking of Musculoskeletal Robots

Abstract

This paper aims to solve the trajectory tracking task of the pneumatic musculoskeletal robot within a model-based reinforcement learning framework. Considering the limited sensors and short lifespan of self-made pneumatic artificial muscles, multi-task Gaussian process regression is employed for micro-data model learning and the learned model is combined with cross entropy method (CEM)-based model predictive control to plan for the optimal action online. To further compensate for the model imperfection and improve the control performance, a proportional derivative controller-like strategy is proposed to guide the search space of the original CEM solver. The effectiveness of our approach is verified on a real musculoskeletal system with one degree of freedom and the results show that only 50 s of interacting with the environment is enough for the robot to learn trajectory tracking skills from scratch.