Attending to Learn and Learning to Attend for a Social Robot

Abstract

Abstract Our motivation is to create a robotic creature, Mertz, that ’lives ’ among us daily and incrementally learns from and about people through long-term social interaction. One of Mertz’s main tasks is to learn to recognize a set of individuals who are relevant to the robot through ongoing human-robot interaction. We present an integrated framework, combining an object-based perceptual system, an adaptive multi-modal attention system and spatiotemporal perceptual learning, to allow the robot to interact while collecting relevant data seamlessly in an unsupervised way. Our approach is inspired by the coupling between the human infants ’ attention and learning process. We implemented a multi-modal attention system for the robot that is coupled with a spatiotemporal perceptual learning mechanism, which incrementally adapts the attention system’s saliency parameters for different types and locations of stimuli based on the robot’s past sensory experiences. We conducted and described results from a six-hour experiment where the robot interacted with over 70 people while collecting various data in a public space. I.