Gradient-Free Deep Q-Networks Reinforcement learning: Benchmark and Evaluation

Abstract

The principles of reinforcement learning present a normative account, firmly related to neuroscience and psychological viewpoints on how animals or humans survive and find their optimal state-action values of an environment. In order to use reinforcement learning successfully in conditions approaching real-world application complexity, however, agents are confronted with a challenging task: they need to obtain correct information of the environment from various kinds of sensors and manage these to generate optimal state-action values from past experience to new conditions and also for long-term survival. Several dedicated approaches have been developed to improve reinforcement learning performances-DQN, Double DQN, SARSA name a few. Since reinforcement learning tasks require maximizing a reward function for the long term, we consider them as challenging optimization problems. Therefore, we use deep Q-network (DQN) that can solve many challenging classic Atari games and robotics problem. However, DQN with BP usually spend a long time to train and difficult to get convergence in short time training. In this paper, we optimize DQN with Genetic Algorithms (GA) for several applications problems. We also provide comparison results between DQN with GA and traditional DQN to show how efficient this method compared to traditional method. The results demonstrate that DQN with GA has better results on each reinforcement learning problem.