Residual Reinforcement Learning Based on Inverse Kinematic Modeling for Soft Robotic Arm Control

Abstract

Abstract Modeling and control of soft robotic arms are challenging due to their complex deformation behavior. Kinematic models offer strong interpretability but are limited by low accuracy, while model-free reinforcement learning (RL) methods, though widely applicable, suffer from inefficiency and require extensive training. To address these issues, we propose a residual reinforcement learning (RRL) modeling and control framework incorporating an inverse kinematic model as prior knowledge to enhance RL training efficiency. Despite the kinematic model producing high mean absolute errors (MAEs) ranging from 33.8 mm to 57.4 mm, it significantly accelerates RL training. Using the Proximal Policy Optimization (PPO) algorithm, our method achieves a 90% reduction in training time and decreases MAEs to 4.8 mm–7.6 mm with just 30,000 iterations. This significantly enhances control precision over inverse kinematic methods while improving efficiency compared to conventional RL approaches.