A Multimodal Scheduler for Synchronized Humanoid Robot Gesture and Speech

摘要

In order to engage in natural and fluent human-robot interaction, humanoid robot companions must be able to produce speech-accompanying non-verbal behavior including hand and arm gestures. In human communication, gestures are considered an integral part of the human thinking process. Accordingly, they are found to be finely synchronized with the accompanying linguistic affiliate [1]. Many researchers have emphasized the importance of this temporal synchrony in terms of coexpressiveness (e.g. [2], [3]). However, for a humanoid robot required to generate speech and gesture, an appropriate synchronization of the two modalities still poses a major challenge. In many existing approaches used for virtual conversational agents or robotic platforms, synchronization of different modalities is either achieved only approximately or by solely adapting one modality to the other, e.g. by adjusting gesture speed to the timing of running speech. Given the limitations of robotic platforms, e.g. motor velocity limits, these approaches turn out to be insufficient when a fine synchronization of speech and gesture is a fundamental necessity for fluent human-robot interaction. We present a multimodal scheduler that is capable of synchronizing expressive hand and arm gestures with speech for the Honda humanoid robot. Since the challenge of multimodal behavior realization has already been tackled in various ways within the domain of virtual agents, our approach exploits the experiences gained from the development of a speech-gesture production model for embodied conversational agents. In particular, we build on the Articulated Communicator Engine (ACE), which is one of the most sophisticated multimodal schedulers and behavior realizers in that it replaces the use of lexicons of canned behaviors with a real-time production of flexibly planned behavior representations [4]. The implementation of the interface that couples ACE with the perceptuo-motor system of the Honda robot and which is now used as an underlying action generation framework for the humanoid is described in [5]. An outline of the implemented robot control architecture is shown in Figure 1.