Robot Sound in Interaction : Analyzing and Designing Sound for Human-Robot Coordination

Abstract

Robots naturally emit sound, but we still know little about how sound can serve as an interface that makes a robot's behavior explainable to humans. This dissertation draws on insights about human practices for coordinating bodily activities through sound, investigating how they could inform robot design. My work builds on three video corpora, involving i) a Cozmo robot in ten family homes, ii) autonomous public shuttle buses in an urban environment, and iii) a teamwork robot prototype controlled by a researcher and interacting with study participants in an experimental setting. I approached the data from two methodological angles, exploring how they can speak to each other: I first carried out an empirical analysis of the video data from an Ethnomethodology and Conversation Analysis (EMCA) perspective, focusing on how humans make sense of robot sound on a moment-by-moment basis in naturally occurring interaction. Subsequently, taking an Interaction Design perspective, I used my video recordings as a design material for exploring how robot sound could be designed in and for real-time interaction. My work contributes to Human-Robot Interaction through detailed studies of robots in the world (rather than in the lab), focusing on how participants make sense of robot sounds. I present a novel framework for designing sound in and for interaction and a prototyping practice that allows practitioners to embed an EMCA stance into their designs. The dissertation contributes to EMCA by describing how members embed autonomous machines into the social organization of activities and how humans treat robots as participants in the interaction. I make a contribution to the development of EMCA hybrid studies by seeking a synthesis between EMCA and robot interaction design. vi Many of my ideas emerged in dialogue with others, and I am incredibly thankful for the many formal and informal conversations that have contributed to this work. Most importantly, I would like to thank my supervisors for embarking on this journey with me. Leelo Keevallik, I cannot thank you enough for your wise guidance, and for leaving me so much freedom to explore within your project. Your curiosity and leadership continue to inspire me. Mathias Broth, you have sparked my initial interest in interaction analysis, and I am so grateful to have you as a co-supervisor. Malte Jung, while