Facial Expression Recognition on Video Data with Various Face Poses Using Deep Learning

Abstract

Facial expressions in humans produce non-verbal communication to convey emotional states in humans; hence, they play an essential role in social interactions between humans. Along with the times, research on facial expression analysis has expanded to automatic facial expression recognition by computers. The facial expression recognition plays a vital role in human-computer interactions, monitoring human behavior, educational techniques, psychological, to sociable robots. In this study, the development of human facial expression recognition was carried out using a deep learning method called You Only Look Once (YOLO) based on Convolutional Neural Network (CNN). There are seven classes of facial expressions that can be recognized, namely angry, disgust, fear, happy, sadness, surprise, and neutral. The datasets used are video-based facial expression datasets such as CK+, IMED, and video data from 8 students of the Informatics Department, Institut Teknologi Sepuluh Nopember (ITS), with various face poses. Based on the experimental results, the best accuracy of the still image dataset is 94% on the CK+ dataset with channel three and learning rate 0.01. Moreover, the accuracy of video data with various face poses achieves 73%.