Human robot interactions using speech synthesis and recognition with lip synchronization

Abstract

The most common way for human interaction to understand each other is through communication with each other, so as in the human robot interaction. The ability to talk is one of the most important technologies in the field of intelligent robotics. When it comes to talk there are two basic type of signal transmitted by people: Auditory and Visual. Speech synthesis and speech recognition are crucial abilities for auditory signal. Likewise, lip synchronization is the key technique for visual signal. The mimic lip synchronization could contribute to the improvement of human robot interaction. In this paper, we design a lip synchronization system for our humanoid robot using Microsoft Speech API (SAPI). With totally thirty degrees of freedom (twelve for mouth) we build sixteen lip shapes to perform all the visemes. With proposed system the precise and mimic lip synchronization can let the users to have favorable impression, and gains the closeness to the people. Finally the whole system is implemented at our humanoid robot head to demonstrate the success of the system proposed herein.