A Deep Reinforcement Learning Control Method for a Four-Link Brachiation Robot

Abstract

Brachiation is a common way for primates to move between treetops. This movement has the characteristics of adapting to complex, discontinuous environment and low energy consumption. However, traditional control methods are often difficult to complete such tasks where the support is point-contact and discrete. Reinforcement learning (RL) gives a solution to such tasks due to its strong ability to adapt to complex environments. Therefore, in this paper, we design a four-link underactuated robot model with a pair of suspended hooks and propose a deep reinforcement learning-based control method to realize its brachiation between bars. The policy for outputting control signals is trained using the proximal policy optimization (PPO) algorithm because of its strong performance and ability to handle continuous action spaces. A reward function considering energy consumption and the challenges posed by the introduction of hooks is designed to obtain an energy-optimized control policy. Through simulation, a comparison is drawn against the author's previously proposed method of generating energy-optimal offline trajectories and tracking by model predictive control (MPC), thereby substantiating the superiority of the proposed method outlined in this paper.