Emotion Recognition from Facial Images using Hybrid Deep Learning Models

Abstract

With the advancement of robots and their assimilation in daily chores, it has become essential to maintain an effective mode of communication between them and humans, that in turn requires development of highly intelligent systems so that robot can sense and adapt the behavior accordingly. The recognition of actual human emotion and its exact interpretation by the machines poses a great challenge to computer vision community, and in quest to improve the adaptivity of machines, a variety of deep learning convolutional neural network (CNN) algorithms have been proposed to serve the purpose. But as Facial Expression Recognition’s (FERs) have found their applications in a number of society verticals like health, education, marketing, gaming, surveillance etc. The single algorithm to provide perfect recognition in all the scenarios has never been established so far; however, the research is still in progress to develop the substitutes or new models to improve the recognition process. In this paper, deep learning algorithms have been utilized for classifying the facial expression of the humans.Two benchmark datasets of facial expression images have been used. The proffered method investigated the effectiveness of DCNN with the help of multiple models. EfficientNetB0, DenseNet169 and a combined model of EfficientNetB0+VGG16 have been proposed to be used in our work.With the hybrid model, we have achieved recognition accuracy of 90.6% and 95.6% on FER2013 and JAFFE datasets respectively. The recognition rates achieved are competitive as compared to previous reported results in literature on the two datasets.