A real-time computational approach for human facial expression recognition based on landmark feature extraction

Abstract

Real-time human facial expression recognition plays a significant role in many application areas, including human-computer interaction, business intelligence, video surveillance, and robotics. Based on facial expressions, computers can interpret human feelings and psychological stages to pro- vide more realistic approximations in real-world applications. This paper proposes a simple but effective solution for real-time Facial Emotion Recognition (FER), using a mask of the most relevant facial features as input data for a machine-learning approach. For this, a compact Con- volutional Neural Network (CNN) classifier associated with a feature extraction layer was used to provide an end-to-end solution that can detect facial expressions from videos with good accuracy rates. The pro- posed approach was validated using a combination of different facial emotion datasets available in the literature, whose precision rates are considerably better than those provided by the state-of-the-art methods. Score rates of 96.83%, 98.58%, and 98.57% were obtained for the JAFFE, RaFD, and CK+ datasets, respectively, indicating that the presented approach is a promising solution for FER in real-time applications.