Incremental Double Q-Learning-Enhanced MPC for Trajectory Tracking of Mobile Robots

Abstract

Achieving precise trajectory tracking for autonomous mobile robots in complex and dynamic environments poses a demanding challenge. In this study, we propose an innovative approach for the online refinement of model predictive control (MPC) through the application of double Q-learning, designated DQMPC. This method harnesses the dynamic interaction capabilities of double Q-learning with operational environment, facilitating the adaptive tuning of MPC parameters to improve the control performance. To enhance the computational real-time performance of the double Q-learning method, we develop an incremental discretization approach that performs nonuniform discretization of the action and state spaces to improve learning efficiency. In addition, we use a time-error-based prioritized experience sampling method to reduce the interdependence between past experiences and thus accelerate the training speed. Through extensive experiments, we validate the effectiveness of our DQMPC method, which consistently outperforms traditional control technologies.