Reinforcement Learning in Continuous State and Action Space

Abstract

To solve complex navigation tasks, autonomous agents such as rats or mobile robots often employ spatial representations.These "maps" can be used for localisation and navigation.We propose a model for spatial learning and navigation based on reinforcement learning.The state space is represented by a population of hippocampal place cells whereas a large number of locomotor neurons in nucleus accumbens forms the action space.Using overlapping receptive fields for both populations, state/action mappings rapidly generalise during learning.The population vector allows a continuous interpretation of both state and action spaces.An eligibility trace is used to propagate reward information back in time.It enables the modification of behaviours for recent states.We propose a biologically plausible mechanism for this trace of events where spike timing dependent plasticity triggers the storing of recent state/action pairs.These pairs, however, are forgotten in the absence of a reward-related signal such as dopamine.The model is validated on a simulated robot platform.