Research on Robot Massage Force Control Based on Residual Reinforcement Learning

Abstract

To address the problem that traditional force control methods have difficulty obtaining stable forces during a robot massage, a robot force control algorithm based on residual reinforcement learning is proposed. An initial strategy of the robot massage force is first constructed with impedance control, but the massage force often fluctuates when the skin environment is unknown to the robot. A reinforcement learning algorithm is then used to analyze the relationship between the robot contact state and offset displacement and compensate for the residuals of the impedance controller. To speed up the search for the compensation strategy, a neural network is constructed to fit a dynamic model of reinforcement learning with the data from the initial strategy, which can simulate the contact between the robot and skin. Offline training in a simulated environment can reduce the number of actual interactions in reinforcement learning and improve the practicability of the algorithm; to integrate the two algorithms, the output of the residual reinforcement learning strategy is smoothed. The experimental results show that the robot force control algorithm based on residual reinforcement learning converges after approximately 80 offline iterations. The force error in the online experiment is basically within ±0.2 N.