A Semi-automated Multi-robot Comedy Performance System with Gesture

Abstract

Abstract Humor is a nuanced field; thus, prior robot comedy efforts have found the stage a relevant and helpful source of HRI analysis data, particularly when multiple performers can interact with each other. However, hand-animating one robot is already high-intensity, and to our knowledge, no one has sought to scale entertainment robot gesture design via domain-specific automation. Thus, this paper aims to: (1) study the use of head gesture through a video analysis of 20 human standup comedians, (2) algorithmically generate robot head gestures for dueling robot comedy scripts based on linguistic analysis, and (3) explore critical features for robot entertainment editing interfaces, such as replaying a scene from the middle of a script during rehearsals, as automation is intended to enhance speed rather than finesse. Human entertainers develop expertise via many hours on the stage, sometimes crashing (failing) or bombing (meeting lackluster response), and other times captivating (success) or ‘killing it’ (high audience response). The value of effective timing and gesture in a range of bi-directional communication scenarios is well established, thus, this work sought to ease the process of creating new multi-robot comedy performances, leveraging a Portable Robot Comedy stage we had developed and deployed with two Blossom robots at a variety of public festivals. Human comedian annotation results discuss how linguistic context can predict best- match gestures, and identify common expressive uses of gesture during standup comedy: positive affect , negative affect , spatial location , and audience interaction . The software analyzes word strings within a script to auto-assign gestures that match the above expressive categories. While this work occurred before Large Language Models became easily accessible, the software is relevant to efficiently adding gesture and time to any generated script. As such, ongoing work extends these efforts to the higher anthropomorphism Pepper robot platform for LLM-human created guided mindfulness meditations.