Hierarchical Free Gait Motion Planning for Hexapod Robots Using Deep Reinforcement Learning

Abstract

This paper addresses the problem of legged locomotion in unstructured environments, and a novel Hierarchical multi-contact motion planning method for hexapod robots is proposed by combining Free Gait motion planning and Deep Reinforcement Learning (HFG-DRL). We structurally decompose the complex free gait multi-contact motion planning task into path planning in discrete state space and gait planning in continuous state space. Firstly, the Soft Deep Q-Network (SDQN) is used to obtain the global prior path information in the Path Planner (PP). Secondly, a Free Gait Planner (FGP) is proposed to obtain the gait sequence. Finally, based on the PP and the FGP, the Center-of-Mass (CoM) sequence is generated by the trained optimal policy using the designed Deep Reinforcement Learning (DRL) algorithm. Experimental results in different environments demonstrate the feasibility, effectiveness, and advancement of the proposed method. Videos are shown at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">http://www.hexapod.cn/hfg-drl.html</uri> .