A Novel Model-Free Actor-Critic Reinforcement Learning Approach for Dynamic Target Tracking

Abstract

Addressing the trajectory tracking problem of a mobile robot in tracking a dynamic target is still one of the challenging problems in the field of robotics. In this paper, we address the position tracking problem of a mobile robot where it is supposed to track the position of a mobile target whose dynamics is unknown a priori. This problem is even more challenging when the dynamics of the mobile robot is also assumed to be unknown, which is indeed a realistic assumption. Most of the trajectory tracking solutions proposed in the literature in the field of mobile robotics are either focused on algorithms that rely on mathematical models of the robots or driven by a overwhelming degree of computational complexity. This paper proposes a model-free actor-critic reinforcement learning strategy to determine appropriate actuator commands for the robot to track the position of the target. We emphasize that mathematical models of both mobile robot and the target are not required in the current approach. Moreover, Bellman's principle of optimality is utilized to minimize the energy required for the robot to track the target. The performance of the proposed actor-critic reinforcement learning approach is backed by a set of computer experiments with various complexities using a virtual circular-shaped mobile robot and a point target modeled by an integrator.