Safe Robot Navigation Using Constrained Hierarchical Reinforcement Learning

Abstract

Safe navigation is one of the steps necessary for achieving autonomous control of robots. Among different algorithms that focus on robot navigation, Reinforcement Learning (and more specifically Deep Reinforcement Learning) has shown impressive results for controlling robots with complex and high-dimensional state representations. However, when integrating methods to comply with safety requirements by means of constraint satisfaction in flat Reinforcement Learning policies, the system performance can be affected. In this paper, we propose a constrained Hierarchical Reinforcement Learning framework with a safety layer used to modify the low-level policy to achieve a safer operation of the robot. Results obtained in simulation show that the proposed method is better at retaining performance while keeping the system in a safe region when compared to a constrained flat model.