Tracking Control for Mobile Robot Based on Deep Reinforcement Learning

Abstract

This paper aims to solve the trajectory tracking problem of mobile robot by using proximal policy optimization (PPO), an advanced deep reinforcement learning algorithm. We adopt a distributed framework of PPO to promote the speed of sample collection and reduce the correlation of transitions when updating the networks. Piecewise random reference state initialization is introduced during training to enable the mobile robot to learn trajectory tracking successfully. In order to promote the sample and training efficiency, we propose a two-stage training strategy which consists of supervised pre-training and fine-training by distributed PPO. Next we introduce LSTM to the actor and critic, and use replay to store the cell state and hidden state of LSTM, which will be used for the initialization of each episode to solve the problem of inaccurate LSTM inital state. We use these different methods to train the mobile robot respectively and the simulation results show that our proposed methods can indeed make some improvements on the performance.