Collaborative Exploration in Human-Robot Teams: What's in their Corpora of Dialog, Video, and LIDAR Messages?

Abstract

This paper briefly sketches new work-inprogress (i) developing task-based scenarios where human-robot teams collaboratively explore real-world environments in which the robot is immersed but the humans are not, (ii) extracting and constructing "multi-modal interval corpora" from dialog, video, and LIDAR messages that were recorded in ROS bagfiles during task sessions, and (iii) testing automated methods to identify, track, and align co-referent content both within and across modalities in these interval corpora. The pre-pilot study and its corpora provide a unique, empirical starting point for our longerterm research objective: characterizing the balance of explicitly shared and tacitly assumed information exchanged during effective teamwork. R 41: I can see in the entrance. C 42: Enter and scan the first room. R 44: I see a door to the right and a door to the left. C 45: Scan next open room on left.