Optimal robot selection by gaze direction in multi-human multi-robot interaction

Abstract

This paper presents a computer vision based system for interaction between multiple humans and multiple robots. Each human can “select” (obtain the undivided attention of) a robot by simply looking directly at it. This extends previous work whereby a single human can select one or more robots from a population. Each robot optimally assigns human identities to tracked faces in its camera view using a local Hungarian algorithm. The gaze-direction and location of the faces are estimated via vision, and a score for each robot-face pair is assigned. Then the system finds the global optimal allocation of robot-to-human selections using a centralized Hungarian algorithm. A useful feature of this method is that robots can be selected by people they cannot see. This is the first demonstration of optimal many-to-many robot-selection HRI.