Deep Reinforcement Learning Based Intelligent Decision Making for Two-player Sequential Game with Uncertain Irrational Player

Abstract

In this paper, two player sequential game with an unknown non-stationary irrational player is investigated for cooperative autonomous robots decision making applications. In practice, the irrationality of agent can seriously degrade the effectiveness of decision making especially for distributed cooperative tasks with applications to multi-robot systems. Specifically, The irrationality can be caused by the cooperation agent's mechanical failure or sensor flaw. To handle this issue, a novel dynamic evaluation system, which includes two important parameters, i.e. cooperation index and competitive flag, is designed to efficiently quantify the player's level of cooperation or competition firstly. Then, the continuous deep Q network space is proposed to predict the action value with respect to a continuous cooperation index. Inspired from the framework of "Friend or Foe" algorithm, a novel hybrid online multi-agent deep reinforcement learning algorithm is proposed. The designed algorithm can evaluate the cooperator's cooperative level as well as maximize the total payoff by learning in a continuous deep Q network space. Eventually, numerical simulation and experimental tests are provided to demonstrate the effectiveness of the designed algorithm.