Reward Functions for Accelerated Learning

Abstract

This paper discusses why traditional reinforcement learning methods, and algorithms applied to those models, result in poor performance in situated domains characterized by multiple goals, noisy state, and inconsistent reinforcement. We propose a methodology for designing reinforcement functions that take advantage of implicit domain knowledge in order to accelerate learning in such domains. The methodology involves the use of heterogeneous reinforcement functions and progress estimators, and applies to learning in domains with a single agent or with multiple agents. The methodology is experimentally validated on a group of mobile robots learning a foraging task. 1 INTRODUCTION Reinforcement learning (RL) has become the methodology of choice for learning in a variety of different domains. Its convergence properties and potential biological relevance make it an approach worth studying. RL has been shown to perform well in Markovian domains, such as games (Tesauro 1992) and simulations (...