Fusing Dynamics and Reinforcement Learning for Control Strategy: Achieving Precise Gait and High Robustness in Humanoid Robot Locomotion*

Abstract

Achieving precise gait planning and high robustness in locomotion control is crucial for the development and application of humanoid robots. In this paper, a novel control strategy is proposed, which combines dynamics control and reinforcement learning (RL), leveraging the precision of dynamics control and the robustness of RL. Specifically, foot placements for each step of the humanoid robot are designed, and the trajectories of the center of mass (CoM) and feet are obtained using a 3D linear inverted pendulum model (3D LIPM). Subsequently, joint angles during motion are calculated based on the trajectories of the CoM and feet using inverse kinematics equations. Finally, the obtained joint angles are trained as baseline actions using RL algorithms. To enhance control robustness, parameter domain randomization is introduced during the training process. By employing this control strategy, simulations of various single-step gaits, such as walking forward, walking to the right, and making right turns, are achieved. Additionally, trajectory tracking, locomotion tests on different terrains, and disturbance resistance are conducted. The simulation results demonstrate that the proposed control strategy enables precise gait control and exhibits strong robustness in humanoid robots.