Finding Cooperation in the N-Player Iterated Prisoner's Dilemma with Deep Reinforcement Learning Over Dynamic Complex Networks

Abstract

Biological, social and economical systems expose enormous levels of complexity, and studying situations of cooperation of conflict encompassing such systems is of particular interest. The N-Player Iterated Prisoner's Dilemma (NIPD) is a general game-theoretic model that captures realistic scenarios of cooperation and conflict. This paper studies the emergence of cooperation in the NIPD, using methods borrowed from the field of reinforcement learning. Such methods attracted immense amounts of attention over the last decade, and have shown promising results in robotics, game-playing, multi-agent systems, etc. However, it is well-known that plain reinforcement learning applied in the NIPD will converge to the Nash equilibrium of the game, which is defection. Therefore, additional mechanisms are needed to foster the development and upkeep of cooperation. This work considers different interconnection topologies for players of the NIPD, and uses dynamic rewiring (players can severe some connections and form others) as a mechanism to encourage cooperation. In such scenarios, the initial interconnection topology is also a crucial aspect. Thus, conducted experiments are concerned with the NIPD over dynamic complex networks, where players can rewire edges in order to position themselves in more cooperative settings. Our results show that cooperation can emerge in such settings.