ALARM: Safe Reinforcement Learning With Reliable Mimicry for Robust Legged Locomotion

Abstract

Legged robots are supposed to traverse complicated environments, which makes it challenging to design a model-based controller due to their functional complexity. Currently, using deep reinforcement learning to improve the adaptability of robots in complex scenarios has been a major research trend. In this paper, we propose Adaptive Latent Aggregation for Reliable Mimicry (ALARM), a reinforcement learning framework that enables safe and robust locomotion in legged robots using only proprioception. This work features a one-step teacher-student training paradigm by constructing an adaptive aggregation strategy, which integrates the merits of imitation learning and reinforcement learning effectively. The framework integrates normalized penalized proximal policy optimization, which penalizes constraint-violating behaviors while optimizing locomotion policy. Our method facilitates efficient sim-to-real transfer, offering a promising approach for real-world legged robot applications.