Facial expression recognition based on global and local feature fusion with CNNs

Abstract

The facial expression recognition is a hot research subject of computer vision, which causes a broad range of application in the domains of human-computer interaction, security and robotics. However, due to the influence of illumination and posture, real-time facial recognition is still a challenge. In this work, a facial expression recognition method based on the parallel convolutional neural network is proposed. Firstly, the face region is extracted from the pre-processed image by the face detector, in accordance with the detected feature points, the face image is cropped into three parts: eyes, noses and mouths. Then, Two CNNs were trained on the original and cropped data sets individually. Finally, the output of the two CNNs is gathered together to complete the classification of the expression. The modified AlexNet, VGGNet and ResNet were used to verify the method on the FER2013 data set. The recognition accuracy of the three models reached 66.672%, 69.407% and 70.744%, respectively. Experiments show that this method has a positive outcome on the recognition of expressions.