Learning Control Under Uncertainty: A Probabilistic Value-Iteration Approach

Abstract

Abstract. In this paper, we introduce a probabilistic version of the well-studied Value-Iteration approach, i.e. Probabilistic Value-Iteration (PVI). The PVI approach can handle continuous states and actions in an episodic Reinforcement Learning (RL) setting, while using Gaussian Processes to model the state uncertainties. We further show, how the approach can be efficiently realized making it suitable for learning with large data. The proposed PVI is evaluated on a benchmark problem, as well as on a real robot for learning a control task. A comparison of PVI with two state-of-the-art RL algorithms shows that the proposed approach is competitive in performance while being efficient in learning. 1