Model-free trajectory tracking control of a 5-DOF mitsubishi robotic arm using deep deterministic policy gradient algorithm

Abstract

This paper introduces a model-free trajectory tracking control framework for a 5-degree-of-freedom (DOF) Mitsubishi RV-2AJ robotic arm, using the deep deterministic policy gradient (DDPG) algorithm. Unlike traditional control methods that depend on precise dynamic models, the proposed approach utilizes DDPG's actor-critic architecture to learn optimal control policies through continuous interaction with the environment, eliminating the need for explicit modeling of the robot's dynamics. Implemented in MATLAB/Simulink with the robotic arm modeled in Simscape, the DDPG agent receives state feedback including joint positions, velocities, and end-effector coordinates and learns to minimize trajectory tracking errors. A reward mechanism inspired by artificial potential fields is designed to encourage efficient convergence to target positions. Simulation results demonstrate that the DDPG-based controller outperforms traditional adaptive neuro-fuzzy inference system (ANFIS) and proportional–integral–derivative (PID) controllers, achieving significant reductions in tracking errors and enhanced robustness in handling complex, nonlinear, and strongly coupled dynamics. These findings underscore the potential of model-free deep reinforcement learning (DRL) approaches in advancing the control of robotic manipulators and autonomous systems.