Evaluating a Hybrid LLM Q-Learning/DQN Framework for Adaptive Obstacle Avoidance in Embedded Robotics

Abstract

This paper introduces a pioneering hybrid framework that integrates Q-learning/deep Q-network (DQN) with a locally deployed large language model (LLM) to enhance obstacle avoidance in embedded robotic systems. The STM32WB55RG microcontroller handles real-time decision-making using sensor data, while a Raspberry Pi 5 computer runs a quantized TinyLlama LLM to dynamically refine navigation strategies. The LLM addresses traditional Q-learning limitations, such as slow convergence and poor adaptability, by analyzing action histories and optimizing decision-making policies in complex, dynamic environments. A selective triggering mechanism ensures efficient LLM intervention, minimizing computational overhead. Experimental results demonstrate significant improvements, including up to 41% higher deadlock recovery (81% vs. 40% for Q-learning + LLM), up to 34% faster time to goal (38 s vs. 58 s for Q-learning + LLM), and up to 14% lower collision rates (11% vs. 25% for Q-learning + LLM) compared to standalone Q-learning/DQN. This novel approach presents a solution for scalable, adaptive navigation in resource-constrained embedded robotics, with potential applications in logistics and healthcare.