The robot shaping approach to autonomous robotics

Abstract

Reinforcement learning (RL) is a direct control method: it doesn't require an explicit model of the system to be controlled. On the contrary, the controller learned using reinforcement learning contains an implicit model of the robot and of its environment. In RL the control system is developed by using as unique source of information a scalar, the so-called reinforcement, which evaluates control actions: the control system receives either positive or negative reinforcements according to the desirability of the state entered as a consequence of a performed action. By using RL it is not necessary to have data to build and validate the system model. Also, only relevant associations between input and output are learned. This is very important when the controlled system is very complex; as we don't know which parts should be modeled in order to control it, we rely on the fact that the states that the controlled system will visit will be with high probability the most interesting and the most frequently visited during the system lifetime. A trainer in charge of providing step-by-step guidance, by means of immediate reinforcements, is added to the system. This set up is what we call robot shaping. (2 pages)