An efficient adaptive input quantizer for resetable dynamic robotic systems

Abstract

An effective class of algorithms used to create autonomous reactive controllers relies only on failure signals from the environment to adapt. These reinforcement algorithms must learn not only the expected long term discounted reinforcement function in the state space, but also search for the control function policy that maximizes it. The computational demand for systems working with continuous functions remains a challenge for real time applications. This paper introduces a state space quantization network that adaptively quantizes the continuous state space into a subset of finite points, Continuous functions are replaced with lookup tables, which are hardware efficient. This paper will first study the advantages and disadvantages of two such quantizers developed by others. It will then consider a network that achieves its quantization effectiveness by exploiting the reset events which take place after failures. Preliminary experiments show efficient learning speeds for a dynamic pole-cart balancer system.