An Audiovisual Child Emotion Recognition System for Child-Robot Interaction Applications

Abstract

We present an audiovisual emotion recognition system tailored to child-robot interaction scenarios. Our proposed system is based on deep learning and the Temporal Segment Networks framework, receives input from both the child's speech and video modalities (the latter represented as RGB and optical flow streams), and tackles several challenges that arise in emotion recognition and child-robot interaction. The system is evaluated on the EmoReact child emotion recognition dataset, significantly outperforming the state-of-the-art on this corpus. In addition, extensive ablation studies are conducted.