Implementation of fuzzy Q-learning based on modular fuzzy model and parallel structured learning

Abstract

In order to realize intelligent agent such as autonomous mobile robots, Reinforcement Learning is one of the necessary techniques in control system. Fuzzy Q-learning is one of the promising approaches for implementation of reinforcement learning function owing to its high ability of model representation. However, in applying fuzzy Q-learning to actual application, the number of iterations for learning also becomes huge as well as almost all Q-learning application. Furthermore convergence performance is often deteriorated owing to its complicated model structure. In this study, implementation method of fuzzy Q-learning is discussed in order to improve the learning performance of fuzzy Q-learning. The modular fuzzy model construction method based on fuzzy Q-learning is proposed in this paper. Multi-grain configuration of modular fuzzy model is compared with parallel structured learning scheme. Through numerical experiments of mountain car task and Acrobot task, I found that the proposed construction of modular fuzzy model improved the performance of fuzzy Q-learning.