Decentralized Multi-Agent Reinforcement Learning Exploration with Inter-Agent Communication-Based Action Space

Abstract

A new challenging area of research in autonomous systems focuses on the collaborative multi-agent exploration of unknown environments where a reliable communication infrastructure among the robotic platforms is absent. Factors like the proximity between agents, the characteristics of the network nodes, and environmental conditions can significantly impact data transmission in real-world applications. We present a novel decentralized collaborative architecture based on multi-agent reinforcement learning to address this challenge. In this framework, homogeneous agents autonomously decide to communicate or not, that is whether to share locally collected maps with other agents in the same communication networks or to navigate and explore the environment further. The agents’ policies are trained using the heterogeneous-agent proximal policy optimization (HAPPO) algorithm and through a novel reward function that balances inter-agent communication and exploratory behaviors. The proposed architecture enhances mapping efficiency and robustness while minimizing inter-agent redundant data transmission. Finally, this paper demonstrates the advantages of the investigated approach compared to a strategy that does not incentivize communicative behaviors.