Practical Reinforcement Learning For MPC: Learning from sparse\n objectives in under an hour on a real robot

Abstract

Model Predictive Control (MPC) is a powerful control technique that handles\nconstraints, takes the system's dynamics into account, and optimizes for a\ngiven cost function. In practice, however, it often requires an expert to craft\nand tune this cost function and find trade-offs between different state\npenalties to satisfy simple high level objectives. In this paper, we use\nReinforcement Learning and in particular value learning to approximate the\nvalue function given only high level objectives, which can be sparse and\nbinary. Building upon previous works, we present improvements that allowed us\nto successfully deploy the method on a real world unmanned ground vehicle. Our\nexperiments show that our method can learn the cost function from scratch and\nwithout human intervention, while reaching a performance level similar to that\nof an expert-tuned MPC. We perform a quantitative comparison of these methods\nwith standard MPC approaches both in simulation and on the real robot.\n