Multi-agent Reinforcement Learning with Biased Experience Sharing in Swarm-robotics Domain

Abstract

Reinforcement learning has been widely applied to solve a diverse set of robot learning tasks. Experience sharing has become an important technique for collaborative multi-robot systems in non-deterministic environments. However, one of the main problems for multi-agent reinforcement learning (MARL) is the large state-action space, which leads to low convergence and even learning failures. This problem is especially true for multi-robot learning with limited communication and mobility. In this paper, a Bilateral Biased Neighbors-Sharing Cooperative Reinforcement Learning (BBNS-CRL) method is presented to accelerate learning process by integrating the neighboring robots' knowledge with local knowledge. The BBNS-CRL method can be applied in collaborative task with multiple homogeneous robots, which have limited communication distance and dynamic neighboring relationship. The value function is shared between two adjacent robots with a biased sharing method. Simulation results on a cooperative multi-robot foraging task show that a better learning convergence can be achieved by adopting BBNS-CRL method.