Real-Time Hierarchical POMDPs for Autonomous Robot Navigation

Abstract

This paper proposes a novel hierarchical representation of POMDPs that for the first time is amenable to real-time solution. It will be referred to in this paper as the Robot Navigation - Hierarchical POMDP (RN-HPOMDP). The RN-HPOMDP is utilized as a unified framework for autonomous robot navigation in dynamic environments. As such, it is used for localization, planning and local obstacle avoidance. Hence, the RN-HPOMDP decides at each time step the actions the robot should execute, without the intervention of any other external module. Our approach employs state space and action space hierarchy, and can effectively model large environments at a fine resolution. Finally, the notion of the reference POMDP, that holds all the information regarding motion and sensor uncertainty is introduced, which makes our hierarchical structure memory efficient and enables fast learning. The RN-HPOMDP has been tested extensively in a real-world environment.