Using spatial language to drive a robot for an indoor environment fetch task

Abstract

This paper proposes a system that allows the use of natural spatial language to control a robot performing a fetch task in an indoor environment. The system processes spatial referencing language and extracts a tree structure of language chunks. The spatial language system is then grounded to a robot navigation instruction in the form of a sequence of actions based on spatial references to furniture and room structure; the best navigation instruction is selected by scoring. In addition, the Reference-Direction-Target (RDT) model is proposed to represent indoor robot actions. To control the robot for the fetch task, a behavior model is designed based on the RDT model. An assistive robot has been designed and programmed based on this system. The proposed spatial language grounding model and robot behavior model are tested experimentally in three sets of experiments. Results show that the system enables a robot to follow spatial language commands in a physical indoor environment even if the referenced furniture items are re-positioned.