Deep Learning Model for Emotion Prediction from Speech, Facial Expression and Videos

Abstract

The rapid development of computer vision and machine learning in recent years has led to fruitful accomplishments in a variety of tasks, including the classification of objects, the identification of actions, and the recognition of faces, among other things. Nevertheless, identifying human emotions remains one of the most difficult tasks to do. To find a solution to this issue, a significant amount of work has been put in. In order to achieve higher accuracy in this reactivity towards a variety of speeches and vocal -based methods, computer intelligence, natural language modelling systems, and other similar technologies have been used. The examination of the emotions has the potential to be useful in a number of different settings. Cooperation with human computers is one example of such a field. Computers can help customers recognize emotions, make wiser decisions, and create more lifelike human-robot interactions. In recent times, there has been a lot of focus placed on the ability to forecast dynamic facial emotion expressions in videos. Therefore, this work proposes a deep convolutional neural networks (CNNs) model for emotion prediction from speech samples, facial expression images, and videos with enhanced prediction accuracy and reduced loss. In addition, the speech CNN model also utilizes mel-frequency Cepstrum coefficients (MFCC) as feature extraction from given speech samples. The proposed MFCC-CNN model resulted in superior performance than traditional models.