Information Sharing for Cooperative Robots via Multi-Agent Reinforcement Learning

摘要

Facilitating collaboration within a team of robots poses a challenging question for the field of multi-agent reinforcement learning (MARL) in smart environments. Many existing cooperative MARL methods utilize centralized or decentralized frameworks leveraging global or local information for decision-making without sufficiently considering information exchange among agents. This research presents an innovative information-sharing approach for MARL, aiming to enhance collaboration among robots and improve overall team performance in multi-agent systems. In particular, our approach introduces an Information Sharing Matrix (ISM) that combines scenario-independent spatial and environmental information with each robot's local observations, thereby enhancing the performance of individual robots and improving their global awareness during the MARL learning process. To assess the efficacy of our approach, we conducted experiments on three cooperative multi-agent scenarios with varying difficulty levels implemented in Unity ML-Agents Toolkit. The experimental results indicate that robots employing our approach have effectively learned collaborative abilities, enabling them to maximize space coverage while avoiding conflicts among themselves. The robots utilizing our ISM-Shared variation outperformed those using decentralized MARL. They achieved performance comparable to robots employing centralized MARL, where complete global information is used for decision-making during the execution. Additionally, our ISM-MARL is adaptable across team sizes and consistently maintains high performance when transferring knowledge to teams of varying sizes, without being explicitly learned during the training phase. This suggests a resilient MARL learning technique that can adapt to changing environments.