Reinforcement learning with guarantees: a review

Abstract

Reinforcement learning is concerned with a generic concept of an agent acting in an environment. From the control theory standpoint, reinforcement learning may be considered as an adaptive optimal control scheme. Despite accumulating evidence of effectiveness of reinforcement learning in various applications, which range from video games to robotics, this control scheme in its bare-bones version provides no guarantees on the performance of the agent-environment closed loop. Measures have to be taken to provide the said guarantees. This survey gives a brief picture of the current progress in this direction. Three major groups of approaches are overviewed: supervisor-based, Lyapunov reinforcement learning and fusion with model-predictive control. The central message of this survey is that a synergy with classical model-based control seems the most promising direction of research in reinforcement learning, as long as it is to become an industry standard.