Designing Visual and Auditory Attention-Driven Movements of a Tabletop Robot

Abstract

This work presents a framework for a visual-auditory attention-driven robot eye-head gaze movement, which combines visual and auditory inputs to determine the direction of gaze movement for a social robot. The framework computes the most salient changes in position by considering both visual and auditory cues. The proposed system was implemented on Haru, a tabletop social robot, where eye-head gaze movement was controlled using visual input from a camera positioned above the eyes and auditory input from a seven-channel microphone. This allowed for eye movement on a two-dimensional flat screen and body rotation towards the person who is speaking. This framework provides a representation of the robot’s attentional gaze that leverages both visual and auditory cues, resulting in more natural and responsive coordinated eye-head gaze movements of the social robot. The potential benefits include improved communication, increased engagement, and a stronger sense of connection with the robot.