Predicting Engagement Breakdown in HRI Using Thin-Slices of Facial Expressions

Abstract

In many Human-Robot Interaction (HRI) scenarios, robots are expected to actively engage humans in interaction tasks for an extended period. We consider a successful robot to be alert to Engagement Breakdown (EB), a situation in which humans prematurely end the interaction before the robot had the chance to receive a complete feedback. In this paper, we present a method for early EB prediction using Echo State Networks (ESNs), a variant of Recurrent Neural Networks. The method is based on Action Units (AUs) of human facial expressions. We apply the proposed architecture to a real-world dataset and show that the architecture accurately predicts EB behavior using 30 seconds of facial expression features.