Instance-Based State Identification for Reinforcement Learning

Abstract

mccallumCcs.rochester.edu This paper presents instance-based state identification, an approach to reinforcement learning and hidden state that builds disambiguat-ing amounts of short-term memory on-line, and also learns with an order of magnitude fewer training steps than several previous ap-proaches. Inspired by a key similarity between learning with hidden state and learning in continuous geometrical spaces, this approach uses instance-based (or "memory-based") learning, a method that has worked well in continuous spaces. 1 BACKGROUND AND RELATED WORK When a robot's next course of action depends on information that is hidden from the sensors because of problems such as occlusion, restricted range, bounded field of view and limited attention, the robot suffers from hidden state. More formally, we say a reinforcement learning agent suffers from the hidden state problem if the