Multi-Lingual Speech Emotion Recognition: Investigating Similarities between English and German Languages

Abstract

Speech emotion recognition is an important research area in speech processing as it has a wide range of applications in fields such as human-robot interaction, sentiment analysis, and mental health monitoring. This paper focuses on emotion recognition in English and German. Using a Convolutional Neural Network (CNN) and the Local Interpretable Model- Agnostic Explanations (LIME) technique, we delve into linguistic and acoustic features crucial for emotion recognition. Our study explores shared and distinct features across languages, illuminating cross-lingual aspects of emotion recognition. Leveraging the IEMOCAP dataset, we applied pitch shifting and extracted Mel Frequency Cepstral Coefficients (MFCC) for dataset robustness. CNN modeling with Adam optimizer, categorical cross-entropy loss, and EarlyStopping ensued. LIME revealed the model’s decisions through feature importance maps. This paper highlights the importance of considering the language and cultural differences in speech emotion recognition for cross-lingual speech analysis.