Emotional McGurk effect - an experiment

Abstract

Video and audio recordings of emotional expressions were used to construct conflicting stimuli in order to perform a McGurk experiment. Perception tests were made on bimodal, auditory, visual and McGurk stimuli. In unimodal condition the auditive channel showed more expected interpretations than the visual channel. In the McGurk condition however, the visual channel was more reliable than audio at conveying emotions. However, most frequently the listener hears something which is present neither in video nor audio. Bimodal non-conflicting information is most reliable. The results are relevant for construction of robots and avatars, since facial emotional expression is always present.