On The Use of Gestures in Dialogue Breakdown Detection

Abstract

In this paper, we verified the use of gestures in dialogue breakdown detection. The data is 30 real human-robot conversations annotated with body, hand and head gestures. We used as features the ratio of the duration of each gesture within the 3000 msec interval from the start of the robot's utterance, and applied Support Vector Machine to build a model to predict whether the robot's utterance had caused dialogue breakdown. The results showed that the accuracy of our model was higher than the chance level in multiple metrics. Moreover, it is suggested that in some cases the users reacted to the robot's response with gestures in the same way as in human-human interaction.