Visual robot homing using Sarsa(λ), whole image measure, and radial basis function

Abstract

This paper describes a model for visual homing. It uses Sarsa(lambda) as its learning algorithm, combined with the Jeffery divergence measure (JDM) as a way of terminating the task and augmenting the reward signal. The visual features are taken to be the histograms difference of the current view and the stored views of the goal location, taken for all RGB channels. A radial basis function layer acts on those histograms to provide input for the linear function approximator. An on-policy on-line Sarsa(lambda) method was used to train three linear neural networks one for each action to approximate the action-value function with the aid of eligibility traces. The resultant networks are trained to perform visual robot homing, where they achieved good results in finding a goal location. This work demonstrates that visual homing based on reinforcement learning and radial basis function has a high potential for learning local navigation tasks.