Four-Legged Gait Control via the Fusion of Computer Vision and Reinforcement Learning

摘要

This article explores the integration of fully autonomous legged robots in obstacle filled environments, simultaneously addressing the challenges of navigation and control. Despite the potential of legged robots for dynamic tasks, their deployment in complex environments has been hindered by the difficulty of developing effective autonomous control systems. In particular, the motion planning problem is addressed in this article, by formulating it as a Partially Observable Markov Decision Process (POMDP) and applying Proximal Policy Optimization (PPO), a model-free Deep Reinforcement Learning (DRL) algorithm. To improve sample efficiency and real-world applicability, the proposed method incorporates a Central Pattern Generator (CPG) for motion planning and a Variational Autoencoder (VAE) for terrain representation, reducing the complexity of action and observation spaces. Referred to as the VAE-CPG architecture, its performance is demonstrated using the Unitree Laikago robot within the PyBullet simulation environment, aiming to show its effectiveness in simulated construction sites. Our findings indicate that by reducing the legged action space to periodic gait patterns and optimizing the gait based on sensory feedback, we achieve enhanced adaptability and efficiency. This work presents a viable means towards the deployment of autonomous legged robots and their improved efficiency in real applications.