Convolution by Multiplication: Accelerated Two- Stream Fourier Domain Convolutional Neural Network for Facial Expression Recognition

Abstract

Facial expression plays an important role in human communication as a type of nonverbal language and has been widely used in various areas such as psychology, human-computer interaction and robotics. Nowadays, convolutional neural network is a promising approach for facial expression recognition. However, convolutional layers can be time-consuming and computationally expensive because a large number of parameters participate in the calculations and need to be updated during training. To improve the performance of deep neural network in facial expression recognition and accelerate training and calculation, we propose a novel framework which adopts efficient element-wise multiplication to replace traditional convolution. To disentangle reliable feature representation for more effective recognition and further enhance the recognition performance while maintaining the efficiency, we propose a representation scheme which can retain informative feature components while removing unreliable ones in Fourier domain based on the proposed multiplication framework. Extensive comparison and ablation studies are conducted on several benchmark datasets, which shows the efficiency and effectiveness of the proposed model.