A Heterogeneous Time-Series Soft Actor–Critic Method for Quadruped Locomotion

Abstract

The locomotion control of unmanned quadruped robots has been one of the greatest challenges in robotics. Deep reinforcement learning has made great achievements in robot control. However, extracting effective features from historical information to improve locomotion agility is still an open and challenging problem. In this paper, a heterogeneous time-series soft actor–critic (HTS-SAC) method is proposed to enable better policy learning from historical data. Firstly, four mutual information decision conditions are developed for feature selection, which can analyze the correlation between input states and motion performance, obtaining the importance of temporal features of different lengths. Then, according to the results of feature optimization, a novel heterogeneous time-series neural network and the HTS-SAC locomotion control method are designed. Finally, the effectiveness of the proposed method is validated on different terrains using a Laikago quadruped robot simulation model.