MARS: Multi-Agent Deep Reinforcement Learning for Complex Environment Exploration

Abstract

Autonomous exploration of complex, unknown environments is a cutting-edge task not entirely solved by the scientific community. When an agent needs to explore a maze without any a priori information about the environment, the lack of proper destinations and explicit task objectives make traditional navigation policies inappropriate. While the literature presents some sporadic deterministic systems able to face the tasks, learning approaches still need an adequate investigation which could prove them to be more suitable and versatile for this purpose. In this paper, we present MARS, a path planner that exploits swarms of robots to optimize the exploration of complex unknown environments, such as mazes. To make the solution scalable, the proposed method exploits two cooperating modules: local and global planners. The local planner is modeled as a Markov Decision Process (MDP) and trained as a Reinforcement Learning (RL) multi-agent system. Each agent has access to image representations of a section of the global map, always centered in the robot reference frame, and decides the next navigation goal to complete the local exploration. The global planner is a deterministic system that recovers the navigation when a local solution is unavailable. The robots share the explored section with peers when they meet in a rendez-vous. We compared our approach to a single deterministic agent, a single RL agent and a close-to-optimal deterministic approach which deploys five greedy agents. The simulation results demonstrate MARS' efficiency, reaching near-optimal levels in significantly less time.