Intelligent control of a quadrotor using reinforcement learning with proximal policy optimization

摘要

Aerial platforms, such as quadrotors, are inherently unstable systems. In several prior works, the task of stabilizing the flight of a quadrotor was approached by different techniques, generally based on classic control algorithms. However, recently, model-free reinforcement learning algorithms have been successfully used for controlling these platforms. In this work, we show the feasibility of applying a reinforcement learning method to optimize a stochastic control policy (during training), to perform the position control of the quadrotor. This process maintains a good sampling efficiency while allowing fast convergence even when using computationally expensive off-the-shelf simulators for robotics and without the necessity of any additional exploration strategy. We used the Proximal Policy Optimization (PPO) algorithm to make the agent learn a reliable control policy. Then, we presented the results of the response of the obtained intelligent controller in several conditions. Additionally, we investigated reward signals based on the Proportional-Integrative-Derivative controller and the possibility of reducing the steady state error of the controller. The experiments for the resultant intelligent controller were performed using the V-REP simulator and the Vortex physics engine and results show that it is possible to train such algorithms to control quadrotors.