Designing a Biped Robot's Gait using Reinforcement Learning's -Actor Critic Method

Abstract

The development of humanoid robots and the improvements in robot-assisted mobility have created a huge need for robots that can learn to walk in challenging environments. Hand engineered approaches might work in constrained environments, but learning-based approaches may prove to be superior owing to their greater generalization. In this work, a simple reinforcement learning technique has been designed for the biped robot's real-time gait planning. The bipedal walking is a process in which the robot constantly engages with its surroundings, evaluates the effectiveness of its control actions based on how it is moving, and then modifies its control approach accordingly. The continuous state space and action space are obtained using the Actor-Critic Model (ACM) reinforcement learning method, which also directly determines the robot's final gait without using the reference gait. In order to decrease the amount of actual robot training, speed up training, and guarantee the acquisition of the final gait, the trained model is transferred to the walking control of the actual robot. Finally, a biped robot is created and constructed to test the viability of the suggested approach. Several experiments demonstrate that the suggested approach can produce the continuous and stable gait planning needed for the biped robot.