Learning stable bipedal locomotion skills for quadrupedal robots on challenging terrains with automatic fall recovery

Abstract

Reinforcement learning has made remarkable strides in advancing quadrupedal locomotion. However, achieving bipedal locomotion for quadrupedal robots remains extremely challenging due to less contact with the surface. Additionally, during the transition from quadrupedal to bipedal locomotion, the body axis shifts from horizontal to vertical, and the center-of-mass rises suddenly. Here, we present TumblerNet, a deep reinforcement learning controller that enables robust bipedal locomotion for quadrupedal robots. Our proposed framework features an estimator that estimates the center-of-mass and center-of-pressure vector and rewards based on this vector, which allows the learning controller to monitor and maintain the balance of the robot during bipedal locomotion. As such, the proposed framework, although only trained on flat ground in simulation, can be directly deployed in a real robot on various terrains without additional training. The proposed framework exhibits exceptional robustness against various challenging terrains (uneven and soft terrains) and external disturbances, with automatic fall recovery.