DRL-Searcher: A Unified Approach to Multirobot Efficient Search for a Moving Target

Abstract

This article studies the multirobot efficient search (MuRES) for a nonadversarial moving target problem, whose objective is usually defined as either minimizing the target's expected capture time or maximizing the target's capture probability within a given time budget. Different from canonical MuRES algorithms, which target only one specific objective, our proposed algorithm, named distributional reinforcement learning-based searcher (DRL-Searcher), serves as a unified solution to both MuRES objectives. DRL-Searcher employs distributional reinforcement learning (DRL) to evaluate the full distribution of a given search policy's return, that is, the target's capture time, and thereafter makes improvements with respect to the particularly specified objective. We further adapt DRL-Searcher to the use case without the target's real-time location information, where only the probabilistic target belief (PTB) information is provided. Lastly, the recency reward is designed for implicit coordination among multiple robots. Comparative simulation results in a range of MuRES test environments show the superior performance of DRL-Searcher to state of the arts. Additionally, we deploy DRL-Searcher to a real multirobot system for moving target search in a self-constructed indoor environment with satisfying results.