Agile DQN: adaptive deep recurrent attention reinforcement learning for autonomous UAV obstacle avoidance

Abstract

Unmanned Aerial Vehicle (UAV) obstacle avoidance in 3D environments demands sophisticated handling of high-dimensional inputs and effective state representations. Current Deep Reinforcement Learning (DRL) algorithms struggle to prioritize salient aspects of state representations and manage extensive state and action spaces, particularly in partially observable environments. Addressing these challenges, this paper proposes Agile DQN (AG-DQN), a novel algorithm that dynamically focuses on key visual features and robust Q-value estimation to enhance learning. The AG-DQN architecture synergizes several components-Glimpse Network, LSTM Recurrent Network, Emission Network, and Q-Network-to dynamically and selectively process crucial visual features, optimizing decision-making without processing the entire state. AG-DQN's adaptive temporal attention strategy also adjusts to environmental changes, maintaining a balance between recent and past observations. Experimental results demonstrate AG-DQN's improved performance over existing DRL methods, highlighting its potential in advancing autonomous UAV navigation and robotics.