Dependable multimodal communication and interaction with robotic assistants

Abstract

To advance research in the field of multimodal human-robot communication we designed and built the humanoid robot Hermes. Equipped with an omnidirectional undercarriage and two manipulator arms it combines visual, kinesthetic, tactile, and auditory sensing with natural spoken language input and output and body expressions for natural communication and interaction with humans. Hermes was successfully tested in an extended six-month experiment in a museum where only naive users interacted with the robot. They chatted with Hermes in several languages and requested various services. Multimodal communication and an understanding of the current situation by the robot turned out to be the key to success.