Enabling humanoid musical interaction and performance

Abstract

Many people incorporate music into their daily lives, and the development of robots with musical awareness provides an opportunity for rich forms of human-robot interaction. Robots must, however, acquire a variety of skills before being able to participate in musical activities. In order to dance or play an instrument, for example, a robot be able to utilize substantial auditory and visual information. Our work focuses on providing such capabilities (audio and visual beat detection, pitch detection, motion control for producing musical notes) with the goal of enabling an adult-sized humanoid to be an interactive participant in a live musical ensemble. Miniature humanoids are used to prototype and refine many of these systems before deploying them on Hubo, an adult- sized humanoid developed by the Korean Advanced Institute of Science and Technology.