Deep Reinforcement Learning for Autonomous Mobile Robot Navigation: Comparing the Performance of DQN, DDPG, and TD3 Algorithms

Abstract

This study evaluates the performance of three deep reinforcement learning algorithms—Deep Q-Network (DQN), Deep Deterministic Policy Gradient (DDPG), and Twin Delayed Deep Deterministic Policy Gradient (TD3)—for autonomous navigation tasks in a complex simulated environment using ROS2 (Robot Operating System 2) and Gazebo with the TurtleBot3 platform. The training was conducted over 10,000 episodes in a combined environment comprising static obstacles, dynamic obstacles, and mazelike layouts. Performance metrics, including success rate, collision rates, tumble rates, and training time were analyzed throughout the training process. The results indicate that TD3 exhibited the highest success rate (83.7%) over the training period, with minimal collision rates (12.5% for static obstacle and 4.1% for dynamic obstacles), and the most efficient path planning. DDPG demonstrated stable learning behavior, achieving a 75.2% success rate while minimizing unnecessary movements and tumbles. Conversely, DQN struggled to adapt to the environment's complexity, recording the lowest success rate (62.5%) and the highest number of wall and obstacle collisions.