Interpreting instruction sequences in spatial language discourse with pragmatics towards natural human-robot interaction

Abstract

We present a methodology for enabling service robots to interpret spatial language instruction sequences expressed through natural language discourse from non-expert users. As part of our approach, we propose a novel probabilistic algorithm for the automatic extraction of contextually and semantically valid instruction sequences from unconstrained spatial language discourse. Additionally, we present the design and implementation details of a procedure for reference resolution of anaphoric expressions encountered within the user discourse. Towards application of our human-robot interaction (HRI) methodology on robot platforms in practice with end users, we discuss a generalized procedure for transfer to physical systems and provide solutions for key pragmatic considerations including the generation of safe robot execution paths for both the robot and people in the environment. The paper concludes with an evaluation of our spatial language-based HRI framework implemented on a PR2 robot to demonstrate the generalizability and usefulness of our approach in real world applications.