Online TD(A) for discrete-time Markov jump linear systems

Abstract

This paper proposes a new approach for the optimal quadratic control of discrete-time Markov jump linear systems (MJLS), inspired on the temporal differences (TD) concepts of reinforcement learning. The method is online, in the sense that it is able to simultaneously apply and refine the currently available controller, and it is transition model-free, because there is no need for explicit knowledge of the Markov chain transition probabilities, provided it can be sampled or simulated. The strategy builds upon a previously proposed offline method and we hope will pave the way for developing and adapting reinforcement learning techniques for MJLS. The method is experimentally evaluated in Samuelson's macroeconomic model and in the control of a faulty robotic manipulator arm, performing favorably when compared to its offline predecessor.