Visual Servoing Gain Tuning by Sarsa: an Application with a Manipulator

Abstract

This paper investigates a Sarsa-based visual servoing control gain tuning method and the application on a manipulator. For a typical visual servo controller, fixed control gains will not provide the best performance. Therefore, state action reward state action (SARSA) algorithm, one of learning-based methods from reinforcement learning (RL), is introduced to select control gains in every control step. The norm of the visual error is used to define the state space. The positive gain of the controller is discretized as the actions. A reward function is defined to evaluate the performance of every action. Both a numerical test and a robot experiment are carried out to validate the presented algorithm.