State generalization method with support vector machines in reinforcement learning

摘要

Abstract A discrete state space is often the subject of conventional reinforcement learning; continuous states must be discretized in order to be handled with conventional learning methods. However, simple discretization increases the state dimensionality and results in an exponential increase in the number of states. The time needed for learning and the memory requirements are therefore greatly increased. In this paper, the authors propose an algorithm for generalizing multidimensional continuous states using a support vector machine (SVM). This algorithm estimates the optimal action in an unknown state using SVM and can be expected to adapt to the environment in a smaller number of trials. To compare this method with conventional algorithms, a simulation experiment was carried out for an assumed task in which a robot is caused to move toward a goal. As a result, it was confirmed that this algorithm adapted to the environment in a smaller number of trials. © 2006 Wiley Periodicals, Inc. Syst Comp Jpn, 37(9): 77–86, 2006; Published online in Wiley InterScience ( www.interscience.wiley.com ). DOI 10.1002/scj.20140