A Review of Multi-Agent Reinforcement Learning Algorithms

Abstract

In recent years, multi-agent reinforcement learning algorithms have demonstrated immense potential in various fields, such as robotic collaboration and game AI. This paper introduces the modeling concepts of single-agent and multi-agent systems: the fundamental principles of Markov Decision Processes and Markov Games. The reinforcement learning algorithms are divided into three categories: value-based, strategy-based, and actor–critic algorithms, and the algorithms and applications are introduced. Based on differences in reward functions, multi-agent reinforcement learning algorithms are further classified into three categories: fully cooperative, fully competitive, and mixed types. The paper systematically reviews and analyzes their basic principles, applications in multi-agent systems, challenges faced, and corresponding solutions. Specifically, it discusses the challenges faced by multi-agent reinforcement learning algorithms from four aspects: dimensionality, non-stationarity, partial observability, and scalability. Additionally, it surveys existing algorithm-training environments in the field of multi-agent systems and summarizes the applications of multi-agent reinforcement learning algorithms across different domains. Through this discussion, readers can gain a comprehensive understanding of the current research status and future trends in multi-agent reinforcement learning algorithms, providing valuable insights for further exploration and application in this field.