Egocentric Visual Locomotion in a Quadruped Robot

Abstract

In recent years, with the rapid development of robotics and computer technology, more and more robots have been integrated into people's lives. In nature, quadrupeds use vision to perform precise and flexible movements. Imitating their vision-based motor skills has been a long-standing challenge in the field of robotics. In this work, we designed an egocentric visual-motor framework that employs techniques such as Long Short-Term Memory Neural Networks (LSTMs), Reinforcement Learning Algorithms (PPOs), and Learning by Cheating to model visual-motor policies. We achieved completely model-free policies by end-to-end training of sensor-action neural networks. Using Unitree Robotics' latest quadrupedal robot, Go2, as a training agent and trained in Nvidia's Isaac Gym simulation environment, the visually-based motion policy showed a 12.43% increase in average survival rate and a 7.6% decrease in the average number of collisions compared to a proprioceptive-only policy. Additionally, the temporally-informative LSTM estimation module exhibited an average survival rate increase of 8.58% and a 9.8% decrease in the average number of collisions compared to the MLP module.