Confidence-Aware Robust Dynamical Distance Constrained Reinforcement Learning for Social Robot Navigation

Abstract

Navigating in a crowded social environment without collisions or freezing is a crucial and challenging task. Recent studies have demonstrated considerable success using Deep Reinforcement Learning for social robot navigation. However, the lack of safety guarantees remains a prevalent problem among these methods. To address this limitation, we present a comprehensive safety framework that builds upon a novel multi-agent environment single-agent decision-making formulation and the strengths of Safe Reinforcement Learning (Safe RL) methods. First, considering the uncertainty of pedestrians and the inaccuracy of trajectory predictions, we explicitly embed confidence-weighted uncertainty and risk measurement in the interaction graph network to induce adaptive policies. Next, we replace the state-wise robust-form unshaped cost with a cumulative robust dynamical distance constraint, which establishes a stronger and essential safety objective while ensuring the invariance of the optimal policy. Then we use reachability-based Safe RL to ensure state-wise safety assurance of the feasible part and the risk minimization of the infeasible part. Simulation and virtual-reality evaluation results demonstrate that our proposed method outperforms the state-of-the-art methods in various metrics, such as success rate and collision energy, achieving the best trade-off between safety and efficiency. Our proposed method yields a policy that exhibits desired collision avoidance behavior in both simulation and real-world experiments.