Reinforcement Learning Based Multi-Layer Bayesian Control for Snake Robots in Cluttered Scenes

Abstract

The majority of current research on reinforcement learning (RL) for snake robot control do not sufficiently account for the spatial and temporal dependencies within the robot or its interaction with its environment during movement. To address this issue, we propose an RL based multi-layer Bayesian method for autonomous snake robot control, which handles challenging scenarios and improves navigation efficiency. There are three major contributions: 1) An innovative hierarchical Bayesian framework unifies gait control, locomotion control, and stimulus reaction; 2) The dynamics of environment is modeled by density propagation and exploited by an LSTM-based agent to improve the learning process; 3) A stimulus reaction model is derived by combining spatial correlation among robot modules and temporal dependency along time sequence. Comparison experiments with a simulated snake robot show that performance of the proposed approach with challenging obstacles is superior to state-of-the-art baseline.