Synthesizing Control Barrier Functions With Artificial Potential Fields for Safe Reinforcement Learning

Abstract

In complex and dynamic environments, achieving autonomous decision-making and control of agent remains a challenging task. Traditional reinforcement learning algorithms often struggle to effectively learn optimal policies when faced with high-dimensional state spaces, sparse rewards, and dynamic obstacles. This article proposes an enhanced deep deterministic policy gradient (DDPG) algorithm. First, we employ an artificial potential field method to pretrain the policy network, providing the reinforcement learning model with a safe initialization capability. This approach significantly reduces early-stage exploration risks and accelerates the convergence process. Furthermore, we integrate control barrier functions (CBFs) into the policy optimization to enhance the ability of dynamic obstacle avoidance, ensuring safety in complex environments. Additionally, we adopt staged rewards, potential-based rewards, and auxiliary rewards to overcome the sparse reward problem, providing the agent with richer and more effective learning signals. In the end, to validate the effectiveness of our designed control scheme, robot operating system Gazebo simulations and practical platform experiments have been conducted. To ensure repeatability, our codes are open sourced on the Github: https://github.com/zhn-ya/DRL.