Learning to Herd Amongst Obstacles from an Optimized Surrogate

Abstract

This paper investigates how a shepherd robot can efficiently steer a coherent group by intelligently moving behind the group in obstacle-filled environments. It was been shown that a model trained by deep reinforcement learning can guide a small number (2–4) of agents among obstacles. However, herding a larger group becomes significantly more challenging because it exhibits the characteristics similar to manipulating a deformable object, i.e., the system is dynamic and the problem is highly underactuated. To overcome these challenges, we show that a model can be trained more effectively via an optimized surrogate, such as a potential field that optimizes the control quality of the group without explicitly considering the placement of the shepherd. Our experiments demonstrate that the trained model is robust to noise for group behaviors and environments. Compared to the rule-based method, the proposed approach maintains a higher probability of guiding the sheep and better control quality.