Does Hard-Negative Contrastive Learning Improve Facial Emotion Recognition?

Abstract

Unconstrained facial emotion recognition has been an active and challenging research over the past decades. Understanding human emotions and enhancing the functionality of human-robot interaction systems depend on the accurate classification of facial expressions. Although the most recent research has concentrated on reducing reliance on a significant amount of clean labeled data, there remains a crucial demand to explore effective representations derived from the available noisy labels in accessible real-world datasets. Therefore, we thoroughly investigate the impact of generalized and transferable latent feature representations on the performance of the facial emotion recognition system. This paper thoroughly analyzes latent feature extraction techniques based on hard-negative contrastive learning. More importantly, we evaluate the benefits derived from utilizing sophisticated feature representations within the fundamental architectural frameworks. We conducted a thorough comparative study on four benchmark datasets, namely FER2013, FERPlus, RAF-DB, and AffectNet. Remarkably, the experimental findings illustrate that the choice of feature representations has a profound impact on facial emotion recognition systems.