Standing balance of single-legged hopping robot model using reinforcement learning approach in the presence of external disturbances

Abstract

In this scholarly investigation, the study focuses on scrutinizing the locomotion and control mechanisms governing a single-legged robot. The analysis encompasses the robot's movement dynamics pertaining to two primary objectives: executing jumps and sustaining equilibrium throughout successive jump sequences. Diverse concepts of this robot model have been scrutinized, leading to the introduction of a distinctive semi-active model devised for maintaining the robot's balance. The research involves an initial design for the robot model followed by the introduction of a multi-phase composite control system. As per the proposed model, the jumping action is facilitated through a four-link mechanism augmented by a spring, while balance preservation is achieved through the independent operation of two arms connected to the upper body. To address the successive jumps within the four-link mechanism, a multi-phase feedback controller is engineered. Additionally, a hybrid control strategy, incorporating the Deep Deterministic Policy Gradient algorithm (DDPG) along with a feedback controller, is proposed to sustain balance throughout the robot's contact and flight phases. The research outcomes, acquired through a series of comprehensive tests conducted within the Simulink simulator environment, demonstrate the robot's capacity to maintain balance over 80 consecutive jumps. The evaluations encompassed various simulated external disturbances, including 1- horizontal impacts on the upper body, 2- disparities in ground height, and 3- alterations in ground angle between consecutive steps. Notably, the findings showcase the robot's adeptness in maintaining balance despite an impact with an amplitude of 25 N for a duration of 0.1 seconds, as well as its resilience in managing ground height disparities up to 3 cm and ground angle variations of up to 3°.