Robotic surveillance and Markov chains with minimal first passage time

Abstract

We propose stochastic surveillance strategies for quickest detection of anomalies in discrete network environments. Our surveillance strategy is determined by optimizing the mean first passage time also known as the Kemeny constant of a Markov chain. We generalize the notion of the Kemeny constant to environments with heterogeneous travel and service times, denote this generalization as the weighted Kemeny constant, and characterize its properties. For reversible Markov chains, we show that both the Kemeny constant and its heterogeneous counterpart can be formulated as convex optimization problems and, moreover, can be expressed as semidefinite programs (SDPs). We numerically illustrate the proposed design: compared with other well-known Markov chains, the performance of our Kemeny-based strategies are always better and in many cases substantially so.