Collision-free robot navigation in confined and partially observable environments using spatial-memory deep reinforcement learning

摘要

Robust navigation is crucial for autonomous robots operating in human-centric environments, but sensor limitations, environmental constraints, and unpredictable human movement make planning under partial observability a challenging task. We propose a deep reinforcement learning-based algorithm that utilizes a spatial-temporal encoding method to address the nonlinear dynamics of robot-crowd interactions, where small motion changes can have a significant impact on the robot’s motion. The model processes the relative positions of humans within the 360∘ sensing range of the robot and integrates a spatial-temporal module built with bidirectional gated recurrent units to retain past observations. An attention mechanism dynamically prioritizes human–robot interactions, while a global planning module within spatial memory ensures both reactive and globally safe path planning in confined spaces. The reward function encourages smooth, collision-free progress, penalizing unsafe actions such as freezing or colliding with walls. Our method, validated in both simulations and real-world experiments, demonstrates its effectiveness in modeling the nonlinear dynamics of robot-crowd interactions through spatial-memory attention and reward shaping, achieving safe and adaptive navigation.