Fast Autolearning for Multimodal Walking in Humanoid Robots With Variability of Experience

Abstract

Recent advancements in reinforcement learning (RL) and humanoid robotics are rapidly addressing the challenge of adapting to complex, dynamic environments in real time. This letter introduces a novel approach that integrates two key concepts: experience variability (a criterion for detecting changes in loco-manipulation) and experience accumulation (an efficient method for storing acquired experiences based on a selection criterion). These elements are incorporated into the development of RL agents and humanoid robots, with an emphasis on stability. This focus enhances adaptability and efficiency in unpredictable environments. Our approach enables more sophisticated modeling of such environments, significantly improving the system's ability to adapt to real-world complexities. By combining this method with advanced RL techniques, such as Proximal Policy Optimization (PPO) and Model-Agnostic Meta-Learning (MAML), and incorporating self-learning driven by stability, we improve the system's generalization capabilities. This facilitates rapid learning from novel and previously unseen scenarios. We validate our algorithm through both simulations and real-world experiments on the HRP-4 humanoid robot, utilizing an intrinsically stable model predictive controller.