Count-based quadratic control of Markov jump linear systems with unknown transition probabilities

Abstract

This paper proposes a model-based approach for the optimal quadratic control of discrete-time Markov jump linear systems (MJLS), in a scenario where the transition probabilities of the Markov chain are uncertain, yet the controller has perfect information of the jump process at all times. We derive an adaptive control strategy that, based on online measurements of the Markov chain, incrementally builds a transition model via maximum-likelihood estimation (MLE) and, at certain time steps, uses it to adjust the current policy in a certainty equivalence fashion. The approach is able to make use of prior information regarding the transition probabilities or, in case this is not available, to address the fully unknown case. Some numerical examples, regarding Samuelson's macroeconomic model and the control of a faulty robotic manipulator arm, illustrate our approach.