Addressing Data Imbalance in Multimodal Conversational Emotion Analysis

Abstract: This research categorized a deep learning based framework for multimodal emotion recognition in conversa- tions, while addressing class imbalance in emotion datasets. The framework combined text, audio, and visual modalities with methods of imbalanced learning to help the recognition of minority emotions. Evaluations across benchmark datasets for the multimodal framework achieved improvements over baseline methods overall, and with respect to the underrepresented classes.

Keywords: Multimodal emotion recognition, unbalanced learning, deep learning