Enhancing Robot Navigation Through Advanced DRL and Cellular Automata Integration

Abstract

Autonomous navigation in dynamic environments poses significant challenges, particularly in enhancing learning efficiency and obstacle avoidance. This paper introduces a novel framework that combines Deep Reinforcement Learning (DRL) methods, including Deep Q-Networks (DQN) and Asynchronous Advantage Actor-Critic with Intrinsic Curiosity Module (A3C-ICM), with Cellular Automata (CA) to address these issues. The integration of CA allows for real-time recalibration of state values, which accelerates the learning process and improves decision-making under complex environmental conditions. Extensive simulation experiments demonstrate that the proposed approach significantly outperforms traditional DRL methods in terms of learning speed, obstacle avoidance, and navigation accuracy. This research contributes to the advancement of autonomous navigation systems by offering a robust, adaptive framework capable of improving robotic performance in challenging, dynamic settings.