Data-Driven Control of Hydraulic Manipulators by Reinforcement Learning

摘要

Motivated by the challenges inherent in achieving high-accuracy tracking control of practical 6-degree-of-freedom (6-DOF) hydraulic robotic manipulators, we aim to conduct research on data-driven control methods in this article. To this end, we introduce actor-critic reinforcement learning to learn the value functions and their corresponding control actions along the trajectories associated with the tracking pattern evolution in hydraulic robotic manipulators. Furthermore, we extend our investigation to encompass the provision of performance guarantees at the system level, even in instances where value functions and control actions are approximated through the utilization of actor-critic neural networks. As proved in this article, the proposed reinforcement learning controller possesses pivotal properties, such as Bellman (sub)optimality in solutions, and the convergence of approximated value functions. We evaluated the proposed reinforcement learning controller in a well-established 6-DOF hydraulic robotic manipulator platform. The experimental results attest to the consistent improvement in control performance, progressively approaching the desired performance benchmarks through iterative updates of control actions. This study makes the first attempt to explore the realization of online adaptive optimal control for hydraulic robotic manipulators within a data-driven paradigm, an area where limited work has been reported, and also sets an example of how reinforcement learning interfaces with industrial automation.