Developing a ContextualizedMultimodal Corpus for Human-Robot Interaction

Abstract

This paper describes the development process of a contextualized corpus for research on Human-Robot Communication. The data have been collected in two Wizard-of-Oz user studies performed with 22 and 5 users respectively in a scenario that is called the Home Tour. In this scenario the users show the environment (a single room, or a whole floor) to a robot using a combination of speech and gestures. The corpus has been transcribed and annotated with respect to gestures and conversational acts, thus forming a core annotation. We have also annotated or linked other types of data, e.g., laser range finder readings, positioning analysis, questionnaire data and task descriptions, that form the annotated context of the scenario. By providing a rich set of different annotated data, the corpus is thus an important resource both for research on natural language speech interfaces for robots and for research on human-robot communication in general. 1.