Mobile robotics planning using abstract Markov decision processes

Abstract

Markov decision processes have been successfully used in robotics for indoor robot navigation problems. They allow the computation of optimal sequences of actions in order to achieve a given goal, accounting for actuator uncertainties. However, MDPs are unsatisfactory at avoiding unknown obstacles. On the other hand, reactive navigators are particularly adapted to that, and don't need any prior knowledge about the environment, but they are unable to plan the set of actions that will permit the realization of a given mission. We present a new state aggregation technique for Markov decision processes, such that part of the work usually dedicated to the planner is achieved by a reactive navigator. Thus some characteristics of our environments, such as the width of corridors, have not been considered, which allows to cluster states together, significantly reducing the state space. As a consequence, policies are computed faster and are shown to be at least as efficient as optimal ones.