Robots Learn Social Skills: End-to-End Learning of Co-Speech Gesture\n Generation for Humanoid Robots

Abstract

Co-speech gestures enhance interaction experiences between humans as well as\nbetween humans and robots. Existing robots use rule-based speech-gesture\nassociation, but this requires human labor and prior knowledge of experts to be\nimplemented. We present a learning-based co-speech gesture generation that is\nlearned from 52 h of TED talks. The proposed end-to-end neural network model\nconsists of an encoder for speech text understanding and a decoder to generate\na sequence of gestures. The model successfully produces various gestures\nincluding iconic, metaphoric, deictic, and beat gestures. In a subjective\nevaluation, participants reported that the gestures were human-like and matched\nthe speech content. We also demonstrate a co-speech gesture with a NAO robot\nworking in real time.\n