Safety-guided Deep Reinforcement Learning for Path Planning of Autonomous Mobile Robots

摘要

The past decade has witnessed the blooming of autonomous mobile robot (AMR) applications with an increasing trend from closed to open working environments with unexpected obstacles. In this context, safety becomes a critical concern in the path planning of AMRs. However, it is a challenging task to guarantee the safety while maintaining the efficiency of path planning algorithm due to the uncertainty of open environments. Therefore, we propose in this paper to categorize the working environment of AMRs into three safety levels, i.e. low-risk, medium-risk and high-risk areas, according to the real-time distance between the AMR and the nearest obstacle. In particular, we incorporate safety levels into the famous reinforcement learning algorithm, deep deterministic policy gradient (DDPG), and develop a safety-guided DDPG algorithm for the path planning of AMRs. In low-risk areas, we adopt the conventional DDPG path planning algorithm directly to guarantee the efficiency (since the safety is generally not an issue in this case). In the medium-risk areas, we design a velocity threshold adjustment method, an OU noise with bias and propose a new reward function, aiming at encouraging AMRs to perform flexible actions as early as possible in order to avoid collisions. In the high-risk areas, we re-design the reward function based on the potential collision risk and shield misleading rewards that may cause local optimum problem, so as to ensure the safety of AMRs in dangerous situations. Simulation results confirm the satisfactory performance of the proposed scheme.