Autonomous Vehicle Drift With a Soft Actor-critic Reinforcement Learning Agent

摘要

Self-driving vehicles have become a more and more important field in recent years. Supported by the techniques of Artificial Intelligence (AI), the current tendency of positive results in applications is making it a promising area to focus further research. Additionally, Reinforcement Learning (RL) is already proved to be an efficient approach for complex problems, e.g. robots, industrial systems and also games (like chess, Go), etc.Drifting is a driving technique at handling limits where the driver intentionally oversteers, with loss of traction, while driving the vehicle through the entirety of a corner. It is a very challenging control task and often results in an accident when it occurs on public roads, consequently, the efficient control of this motion is especially important in the safety of autonomous vehicles.The paper reports novel research results whose main goal is to develop a self-driving agent for drift motion control based on vehicle simulation by Matlab/Simulink. Longitudinal and lateral velocity together with the yaw rate formed the state representation of the vehicle. The agent action space consists of two continuous actuator values: pedal ratio and roadwheel angle. The goal of the agent is twofold: first, it has to jump into a drifting state, second, it has to keep the vehicle in drift.The simulation results show that the proposed Soft Actor-Critic (SAC) RL agent is capable of learning to approach a pre-determined drift equilibrium from cornering and staying in this drift situation as well. For the training, the solution excluded using any kind of prior data, it only works with information gained from the simulation model, which is a remarkable difference from the actual state-of-the-art RL-based solutions.