Trajectory State Model-based Reinforcement Learning for Truck-trailer Reverse Driving

Abstract

Maneuver the truck-trailer wheeled robot (TTWR) system for reverse trajectory following is a challenging task even for experienced drivers, which requires advanced control strategies to ensure system stability, steering precision, and driving safety. In this paper, we present the integration of a small size network model with trajectory planning and reference state prediction to achieve remarkable sample efficiency compared with pure model-free reinforcement approaches. By employing the trajectory state prediction model as the foundation model to interact with the Proximal Policy Optimization (PPO) framework, we enable reversible access to the Markov Decision Process (MDP) dynamics and help the PPO to better estimate vehicle dynamics used for controller. A numerical simulation is conducted to demonstrate the trajectory following accuracy of the proposed methodology compared with popular industry control methods such as linear-quadratic regulator (LQR). Our results indicate that the proposed hybrid trajectory model based approach not only reduces the need for extensive data collection but also achieves similar control accuracy, suggesting a promising direction for future research in autonomous vehicle control, emphasizing the need for efficient, adaptable, and robust learning algorithms.