Reinforcement Learning of Robotic Motion with Genetic Programming, Simulated Annealing and Self-Organizing Map

Abstract

Reinforcement learning, a sub-area of machine learning, is a method of actively exploring feasible tactics and exploiting already known reward experiences in order to acquire a near-optimal policy. The Q-table of all state-action pairs forms the basis of policy of taking optimal action at each state. But an enormous amount of learning time is required for building the Q-table of considerable size. Moreover, Q-learning can only be applied to problems with discrete state and action spaces. This study proposes a method of genetic programming with simulated annealing to acquire a fairly good program for an agent as a basis for further improvement that adapts to the constraints of an environment. We also propose an implementation of Q-learning to solve problems with continuous state and action spaces using Self-Organizing Map (SOM). An experiment was done by simulating a robotic task with the Player/Stage/Gazebo (PSG) simulator. Experimental results showed the proposed approaches were both effective and efficient.