Task-specific pre-learning to improve the convergence of reinforcement learning based on a deep neural network

Abstract

The poor convergence of reinforcement learning based on neural networks is of great challenging for autonomous robots. Inspiring from human reinforcement learning, we propose a two-phase reinforcement learning model based on deep neural networks to improve the convergence problem. In phase 1, task-specific pre-learning based on supervised learning is employed to train a nonlinear neural network in order to approximate a strong reward function. In phase 2, the learned neural network is modified and expanded to a deep neural network to realize reinforcement learning. We test this two-phase approach on the famous mountain-car problem and an autonomous movement controller of a wheeled robot by simulation. The experimental results show that task-specific pre-learning can significantly improve the convergence of traditional reinforcement learning based on neural networks.