Continuous valued Q-learning for vision-guided behavior acquisition

Abstract

Q-learning, a most widely used reinforcement learning method, normally needs well-defined quantized state and action spaces to converge. This makes it difficult to be applied to real robot tasks because of poor performance of learned behavior and a further problem of state space construction. This paper proposes a continuous valued Q-learning for real robot applications, which calculates the contribution values for estimating a continuous action value in order to make motion smooth and effective. The proposed method obtained a better performance of desired behavior than the conventional real-valued Q-learning method, with roughly quantized state and action. To show the validity of the method, we applied the method to a vision-guided mobile robot of which the task is to chase a ball. Although the task was simple, the performance was quite impressive. A further improvement is discussed.