Coordinating Search with Foundation Models and Multi-Agent Reinforcement Learning in Complex Environments

Abstract

We present a multi-agent system simulation designed for efficient coordination and collaboration among multiple robots, particularly suited for search operations. This simulation reflects unstructured and complex outdoor scenarios where significant obstruction and occluded terrain surfaces cause difficulties in search. The software integrates well with reinforcement learning (RL) and a centralized Multi-Agent Transformer (MAT) to enable autonomous robots to collect, process, and integrate data. Search, coverage, and complex mobility planning can be tested in this simulation with dynamic and unstructured environments. The project code and videos can be found at https://github.com/DIRECTLab/Coordinating-MAT-Env