The Effectiveness of State Representation Model in Multi-Agent Proximal Policy Optimization for Multi-Agent Path Finding

摘要

Multi-agent pathfinding plays a crucial role in various robot applications. Recently, deep reinforcement learning methods have been adopted to solve large-scale planning problems in a decentralized manner. Nonetheless, such approaches pose challenges such as non-stationarity and partial observability. In this paper, we address these challenges by integrating a state representation model into a multi-agent proximal policy optimization framework. To do so, we propose to utilize a state representation model which extracts representation features from the global map and leverages this information to enhance the training process. Our approach involves decoupling the feature extractor from the agent training process, enabling a more accurate representation of the global state that remains unbiased by the actions of the agents. Furthermore, our modularized approach offers the flexibility to replace the representation model with another model or modify tasks within the global map, without the retraining of the agents. We demonstrated the effectiveness of our approach by comparing three multi-agent proximal policy optimization frameworks. Our experimental results demonstrate that our approach improves the average episode reward compared to the other approaches.