Multisensory integration effect of humanoid robot appearance and voice on users’ affective preference and visual attention

Abstract

Appearance and voice are essential factors impacting users’ affective preferences for humanoid robots. However, little is known about how the appearance and voice of humanoid robots jointly influence users’ affective preferences and visual attention. We conducted a mixed-design eye-tracking experiment to examine the multisensory integration effect of humanoid robot appearances and voices on users’ affective preferences and visual attention. The results showed that the combinations of affectively preferred voices and appearances attracted more affective preferences and shorter average fixation durations. The combinations of non-preferred voices and preferred appearances captured less affective preferences and longer fixation durations. The results suggest that congruent combinations of affectively preferred voices and appearances might motivate a facilitation effect on users’ affective preference and the depth of visual attention through audiovisual complements. Incongruent combinations of non-preferred voices and preferred appearances might stimulate an attenuation effect and result in less affective preferences and a deeper retrieval of visual information. Besides, the head attracted the most amount of visual attention regardless of voice conditions. This paper contributes to deepening the understanding of the multisensory integration effect on users’ affective preferences and visual attention and providing practical implications for designing humanoid robots satisfying users’ affective preferences.