Solving Nonlinear Continuous State-Action-Observation POMDPs for Mechanical Systems with Gaussian Noise

Abstract

In this paper, we introduce a novel model-based approach to solving the important subclass of partially observable Markov decision processes (POMDPs) with Gaussian noise in contin-uous states, actions, and observations. This kind of POMDP frequently appears in robotics and many other real-world control problems. However, except for the linear quadratic Gaus-sian case, no efficient ways of computing optimal controllers are known. We propose a novel method for efficiently approximating optimal solutions of nonlinear stochastic continuous state-action-observation POMDPs in high dimensions. We use Gaussian processes (GPs) to model both the latent transition dynamics and the measurement mapping. By explicit marginalization over the GP posteriors our method is robust to model errors and can be used for principled belief space inference, policy learning, and policy execution.