Modeling Human-Robot-Interaction based on generic Interaction Patterns

Abstract

Interaction provides many benefits for the control of robot systems, not only in domains in which it is the primary function of the robot to interact socially with people, but also in domains in which the robot requires peer-to-peer interaction skills for solving a specific task in collaboration with the human [Fon03].Speech is the most natural access to the robot's capabilities, and -in contrast to traditional teleoperation -Human-Robot-Interaction is two-way [FTB01]: The user can specify tasks, on the execution of which the robot can provide feedback, both participants can convey information and ask questions -and all that while being busy with another task [CO95].Nevertheless, dialog management is slightly neglected in robotics.Even the Human-Robot-Interaction (HRI) community either focuses on single aspects of interaction, such as the role of gaze or the robot's external appearance, or uses very simple approaches for dialog management that exhibit a number of shortcomings.As an example, at the ACM/IEEE HRI 2011 conference, which is the one of the most recognized and most selective conferences in the field, only four out of 13 contributions presented systems that were autonomous.Of those four, only one system involved an interaction with the robot at all [SWWB11], but it did not make use of a dedicated dialog system.At the AAAI Dialog with Robots 2010 Symposium, which was explicitly dedicated to dialog modeling on robots, 15 out of 21 interactive robots were autonomous.From those, only seven relied on approaches that can be characterized as generalizable [COB10, IST + 10, NIN + 10, PW10a, RS10, Ros10, SIKN10], while the others relied on very specific or scripted solutions.At the same time, existing approaches for dialog management in traditional domains, such as information retrieval, do not fully take the special nature of Human-Robot-Interaction into account.This position (which will be further detailed throughout this thesis) may seem surprising, given the fact that approaches exist that consider phenomena like turntakingCompared to modeling those subtleties, modeling interaction through which a robot can be taught or instructed appears almost trivial.But let us consider a simple fetch-and-carry task in which a mobile robot companion is instructed to fetch an object from another room, e.g. from the kitchen.Besides the skills directly related to the task, such as face recognition, object recognition and navigation skills, this scenario requires interaction skills that go beyond mere understanding of the human's instruction.For example, if the object the robot is supposed to find is not in its usual place, or if an obstacle is blocking 1 Of course, traditional dialog systems are not completely stand-alone either, but have some sort of application back-end as well.The back-end is, however, typically less autonomous than in robotics, and communication is often based on a master-slave relationship.Chapter 3.4.4presents the concept of Interaction Patterns, which serve both as internal dialog model and as developer's application programming interface (API).Details about the dialog management process and aspects such as global discourse management, grounding and multimodality, are provided.Chapter 4 presents the developer-centered evaluation of the approach.Framework efficacy is addressed by discussing four case studies in which the approach is compared with existing, well-established approaches.Framework usability is investigated through a usability test. Part 2: Applications of the Proposed ApproachChapter 5 outlines the iterative development process of the novel approach to dialog modeling and gives an overview of all scenarios that were implemented within the context of this work, either with the suggested PaMini (Pattern-based Mixed Initiative) framework or with its predecessors.Chapter 6 describes two preliminary scenarios that the author contributed to, namely the Home-Tour scenario and different iterations of the Curious