Risk-Sensitive Autonomous Exploration of Unknown Environments: A Deep Reinforcement Learning Perspective

Abstract

This study provides a thorough investigation into autonomous exploration within unknown environments, with a focus on minimizing exploration time and so fuel consumption. This research utilizes a 2D simulation environment to collect training data efficiently, facilitating the evaluation of the proposed methods’ efficiency, adaptability, and generalizability through various experiments. Single-robot autonomous exploration policies using advanced Deep Reinforcement Learning (DRL) algorithms are developed. The main novelty of this paper is the development of risk-sensitive policies, in contrast to traditional risk-neutral approaches in DRL, to enhance exploration efficiency. Additionally, this research presents the development of an adaptive autonomous exploration policy that dynamically adjusts the Conditional Value-at-Risk (CVaR) based on the exploration percentage. The results demonstrate a significant improvement in autonomous exploration efficiency compared to well-known traditional RL and classical single-robot exploration policies, validating the effectiveness of the suggested novel autonomous exploration strategies.