Ramp Jump Control of Single-track Two-wheeled Robot using Reinforcement Learning with Demonstration Data

Abstract

The single-track two-wheeled (STTW) robot ramp jump is a challenging task. This task not only requires the STTW robot to maintain its balance during the various phases of the jump but also requires it to use different speeds for different ramp terrains. This article proposes a deep reinforcement learning method to generate a controller for the STTW robot to complete the ramp jump task. First, the corresponding state space and action space are designed based on the characteristics of ramp jumping. Second, an effective reward function is developed to help train the agent. Then, the demonstration data are introduced into the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm to obtain better training results. Finally, the proposed control method is verified in a dynamic simulation environment. Simulation results show that the proposed control method can enable the STTW robot to complete the ramp jump task on the training terrain and on some other test terrains.