Controlling Soft Robotic Arms Using Hybrid Modelling and Reinforcement Learning

Abstract

Soft robotic arms exhibit high deformability and degrees of freedom, which brings challenges in modeling accuracy and susceptibility to gravitational effects, resulting in imprecise control. This study proposes a reinforcement learning control strategy based on a hybrid model for precise control of soft robotic arms. The hybrid model is formulated by gathering a small dataset of analytical modeling errors of coordinates under various loading (0 g<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\sim$</tex-math></inline-formula>300 g) conditions, followed by employing a Multilayer Perceptron (MLP) to fit these errors, thereby reducing the forward kinematics MAEs (mean absolute errors) from a range of 75.2 mm<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\sim$</tex-math></inline-formula>95.3 mm to 5.9 mm<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\sim$</tex-math></inline-formula>9.1 mm. The reinforcement learning virtual environment is built upon the hybrid model and the Proximal Policy Optimization (PPO) algorithm is used to train control policies. The efficiency of the control policy is validated for different trajectories and loading conditions both in simulation and on the soft robotic arm prototype, revealing distance errors of 3.9 mm<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\sim$</tex-math></inline-formula>6.7 mm and 12.1 mm<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\sim$</tex-math></inline-formula>15.4 mm, respectively, representing 1.1%<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\sim$</tex-math></inline-formula>1.9% and 3.5%<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\sim$</tex-math></inline-formula>4.4% of the total arm length.