DDPG Reinforcement Learning Experiment for Improving the Stability of Bipedal Walking of Humanoid Robots

Abstract

To improve the stability of bipedal walking of humanoid robots, we developed a method of setting trajectory parameters using reinforcement learning on a treadmill like testbed in a real-world environment. A deep deterministic policy gradient (DDPG) was used as the reinforcement learning algorithm. By improving the reward using a zero moment point (ZMP), the optimum value of walking stability and walking speed was determined. The robot was designed to measure the ZMP and mount weights on the upper body. In addition, a treadmill was manufactured to operate at the same speed as the walking speed of the robot. Reinforcement learning was divided into unweighted cases and cases with a weight of 1kg. At approximately 100 min, 300 episodes were performed, and reward improvements of 16.71% and 26.25% reward improvements were made. The ZMP measurements indicated that bipedal walking was performed in a safe area. Therefore, we demonstrated that the biped walking performance of a humanoid robot can be improved by the reinforcement learning of walking speed and ZMP similarity.