Multimodalcues for addressee-hood in triadic communication with a human information retrieval agent

摘要

Over the last few years, a number of studies have dealt with the question of how the addressee of an utterance can be determined from observable behavioural features in the context of mixed human-human and human-computer interaction (e.g. in the case of someone talking alternatingly to a robot and another person). Often in these cases, the behaviour is strongly influenced by the difference in communicative ability of the robot and the other person, and the "salience" of the robot or system, turning it into a situational distractor. In the current paper, we study triadic human-human communication, where one of the participants plays the role of an information retrieval agent (such as in a travel agency where two customers who want to book a vacation, engage in a dialogue with the travel agent to specify constraints on preferable options). Through a perception experiment we investigate the role of audio and visual cues as markers of addressee-hood of utterances by customers. The outcomes show that both audio and visual cues provide specific types of information, and that combined audio-visual cues give the best performance. In addition, we conduct a detailed analysis of the eye gaze behaviour of the information retrieval agent both when listening and speaking, providing input for modelling the behaviour of an embodied conversational agent.