CuriousRL: Curiosity-Driven Reinforcement Learning for Adaptive Locomotion in Quadruped Robots

摘要

Though Proximal Policy Optimization (PPO) has emerged as a dominant algorithm for quadruped locomotion due to its stability and ease of implementation, its learning efficiency is affected by a limited exploration ability of the algorithm. We combine PPO with the Intrinsic Curiosity Module (ICM) to form CuriousRL, which enhances the exploration aspect of PPO, making the quadruped locomotion autonomous and adaptive in dynamic environments. ICM provides intrinsic rewards to the robot in addition to the external environmental rewards from PPO, fostering exploration. We use CuriousRL to teach quadruped robots learn to walk themselves autonomously. We simulate the experiments in Isaac Gym using the ANYmal quadrupeds and measure the performances in dynamic test environments with obstacles and uneven terrains using various environment sensor data including positions, velocities, forces, and torques in the legs and joints. We illustrate that CuriousRL performs better in terms of exploring effective policies and avoiding risk-averse stationary policy adaptation and ehancing sample efficiency.