Autonomous Gain Tuning for Differential Drive Robots Targeting Control using Soft Actor-Critic

Abstract

Differential drive robots (DDRs) belong to a unique category of mobile robots that regulate their speed and direction by independently adjusting the speeds of two wheels. Due to their high maneuverability, DDRs can execute various missions requiring precise positioning and navigation. To guide DDRs in target tracking, the Approximate Pose Increment Control (APIC) is applied to provide reference direction and speed control based on the Line of Sight (LOS) guidance principle. However, the ordinary APIC is unable to consider the physical constraints on the DDRs, potentially resulting in poor tracking performance when the target is near the DDR. One of the most common failure scenarios is the "deadlock loop", which prevents DDRs from reaching the target and causes them to keep circling near it due to a too-large turning radius. To address this issue, the Reinforcement Learning (RL) based APIC is proposed, allowing the system to learn optimal actions from the environment to reach the goal. In this approach, a Soft Actor-Critic (SAC) agent is trained to dynamically adjust two gain values in APIC based on real-time observations. The proposed method not only enhances the targeting performance of APIC but also provides an expert guidance law for imitation learning.