Reinforcement learning of sensor-based reaching strategies for a two-link manipulator

Abstract

This paper presents a neural controller that learns goal-oriented obstacle-avoiding reaction strategies for a multilink robot arm. It acquires these strategies through reinforcement learning from local sensory data. The robot arm has rings of range sensors placed along its links. The neural controller achieves a good performance quite rapidly and shows good generalization abilities in the face of new environments. Suitable input and output codification schemes help greatly to attain these aims. The input codification exploits the inherent symmetry of the robot kinematics and the action given by the controller is interpreted with regard to the shortest path vector (SPV) to the closest goal in the configuration space. In order to avoid the SPV computation for multilink manipulators, we put forward the use of a module for differential inverse kinematics based on the inversion of a neural network that has been previously trained to approximate the manipulator forward kinematics. The use of this module does not only get round the SPV calculation, but also speeds up the learning process.