Design of an entertainment robot with multimodal human-robot interactions

Abstract

The paper presents the design of an entertainment robot focusing on development of multiple modalities of human-robot interactions. The robot system integrates a 16-bit cost-effective embedded MCU, SPCE061A, and a variety of sensory input/output systems to fulfill multimodal interactions. First, it utilizes the ADC and DACs hardware units and the speech processing software library of SPCE061A to realize an interactive dialogue system with speech recognition capability. Next, it combines an assembled head with several facial feature components and control the movements of the head and the facial features to express several head gestures and facial expressions. Furthermore, it arranges an array of microphone modules surrounding the robot body to detect the direction of sound sources according to sequence of arrival times. Finally, it incorporates the afore-mentioned modalities of interactions to provide several behaviors of the entertainment robot, such as singing, crying, laughing and looking around, correspondent to different events.