Improving Multiagent Actor-Critic Architectures, with Opponent Approximation and Dropout for Control

Abstract

In the domain of reinforcement learning, solution proposals to multiagent problems are evolving. We propose a new algorithm, MADDPGX, to handle the problem of higher uncertainty created by other agents' actions by an enemy actor approximator, and we investigate the most efficient techniques of estimations. This approximation works using a neural network, which has the input of the state and the output as the action (probably preferred by the enemy agent). We also experimented with dropout, a tool commonly used for neural networks, but has not been used efficiently for reinforcement learning until now. We have also found that in multiagent actor-critic scenarios, it can improve overall performance. Generally, our contribution is the use of action approximation of adversaries and the dropout usage in actor-critic systems, with a conclusion that the newly proposed methods will perform better in zero-sum multi-agent robot system scenarios. The experiments were conducted in a multiagent predator-prey environment.