Learning Distributed Cooperative Policies for Security Games via Deep Reinforcement Learning

Abstract

A rich amount of literature is available for solving the problem of finding equilibrium strategies in two-player security games that harness the power of integer linear programming (ILP). However, in practice, most security games are accurately modeled with multiple agents where ILP methods either fail to find the optimal solution or the state space is large enough making ILP methods an impractical solution. In this paper, we consider a multi-agent security game setting and propose MultiOptGrad: a novel deep reinforcement learning-based solution to learn distributed optimal policies for defenders. Additionally, using MultiOptGrad we built an reinforcement learning framework for robotic bodyguards that recommend deployment strategies for them in a coordinate system. To demonstrate the effectiveness of our proposed solution, we consider an urban security game where a team of robotic bodyguards are protecting a VIP from physical assault in the presence of neutral and/or adversarial bystanders. Our empirical analysis has shown that MultiOptGrad outperformed quadrant load-balancing (QLB): a hand-engineered technique for solving the VIP protection problem.