Estimation of Speaker’s Confidence in Conversation Using Speech Information and Head Motion

Abstract

Research and development of dialog robots for the purpose of conversation has been actively conducted in recent years. Many of these robot technologies are specialized in language information such as speech recognition and sentence comprehension. However, the majority of the information that human beings emphasize during dialog is nonverbal information. Even in the same words, the meaning of words changes as voice tones, expressions and gesture change. In order for a robot to realize a smooth dialog like a human being, it is important to combine language information and nonverbal information. Therefore, this paper focus on speech information and head motion to estimate the confidence of the user. The confidence conditions were defined as five-level status according to the questionnaire answer: most confident, comparative certainty, neutral, relative uncertain, most unconfident. Two experiments were conducted in this paper. The first experiment was conducted machine learning by summarizing all data of participants. The second experiment was conducted machine learning for each participant with increasing the number of data per participant. The best results could be classified as the most confident/unconfident with an accuracy of 91.3% analyzed by multilayer perceptron (MLP) via Weka. Accuracy increased by about 31% over the first experiment. This is considered to be the reason that the characteristics of individuals were manifested by doing machine learning with data of each participant.