Continuous Action Generation of Q‐Learning in Multi‐Agent Cooperation

Abstract

Abstract Conventional Q ‐learning requires pre‐defined quantized state space and action space. It is not practical for real robot applications since discrete and finite numbers of action sets cannot precisely identify the variances in the different positions on the same state element on which the robot is located. In this paper, a Q ‐Learning composed continuous action generator, called the fuzzy cerebellar model articulation controller ( FCMAC ) method, is presented to solve the problem. The FCMAC displays continuous action generation by linear combination of the weighting distribution of the state space where the optimal policy of each state is derived from Q ‐learning. This provides better resolution of the weighting distribution for the state space where the robot is located. The algorithm not only solves the single‐agent problem but also solves the multi‐agent problem by extension. An experiment is implemented in a task where two robots are taking action independently and both are connected with a straight bar. Their goal is to cooperate with each other to pass through a gate in the middle of a grid environment.