Distributed Coach-Based Reinforcement Learning Controller for Snake Robot Locomotion

Abstract

Reinforcement learning commonly suffers from slow convergence speed and requires thousands of episodes, which makes it hard to be applied for physical robotic applications. Little research has been studied for snake robot control using RL because of the additional difficulty of high redundancy of freedom. Existing methods either adopts an asynchronous A3C structure or a joint state representation. We propose a distributed coach-based deep learning method for snake robot control, which can greatly expedite the training speed with less episodes. The major contributions include: 1) a completely distributed graphical formulation; 2) an explicit stochastic density propagation rule for each robot link; 3) various interaction models with uncertainty estimation. The preliminary results of both simulation and real-world experiments have demonstrated the promising performance in comparison with state-of-the-art.