Adaptive reinforcement Q-Learning algorithm for swarm-robot system using pheromone mechanism

Abstract

The states and actions of the robots in uncertain environments are continuous, which will easily lead to the problem of slow learning speed and the combinatorial explosion issue of the reinforcement learning. Ant colony optimization (ACO) is an evolution algorithm based on swarm mechanism that takes full advantage of the pheromone mechanism to simplify the information sharing and collaborative issues between the swarm individuals. Adaptive robust reinforcement Q-Learning algorithm based on ACO is proposed from two parts: adaptive discretization part and pheromone part. Firstly, adaptive discretization of the continuous input space is realized by the self-organizing neural network. Secondly, the pheromone mechanism of ACO is introduced to improve the traditional reinforcement learning process, which can improve the adaptive capabilities of the system and reduce the space complexity of accelerating the learning speed of the swarm robots. Player/Stage is used as the simulation platform to verify the proposed algorithm. The results show proposed algorithm has efficiency and adaptive capacity in the swarm robotic system.