Dynamic correlation matrix based multi-Q learning for a multi-robot system

Abstract

Multi-robot reinforcement learning is a very challenging area due to several issues, such as large state spaces, difficulty in reward assignment, nondeterministic action selections, and difficulty in merging learned experiences from other robots. In this paper, we propose a dynamic correlation matrix based multi-Q learning (DCM-MultiQ) method for a distributed multi-robot system. A novel dynamic correlation matrix is proposed, which not only handles each agentpsilas Q value, but also deals with the correlation among agents. Furthermore, a theoretical proof of the convergence of the proposed DCM-MultiQ algorithm is also provided using a feedback matrix control theory. To evaluate the efficiency of the proposed DCM-MultiQ method, several case studies of a multi-robot system in forage tasks have been conducted. The simulation results show the efficiency and convergence of the proposed method.