Handling stochastic reward delays in machine reinforcement learning

Abstract

The main contribution of this work is a novel learning algorithm for machine reinforcement learning when Poissonian stochastic time delays are present in the reinforcement signal. The novel approach can deal with rewards which may be received out of order in time or overlap with one another. A PID controller is simulated with and without a stochastic time delay to demonstrate the difficulties of the problem. Experimental results with mobile robots demonstrate that the proposed method improves the performance over that of traditional Q-learning for a learning agent in an environment with Poissonian-type stochastically delayed rewards.