关键词: 多模态情感分析, 图文情绪分类, 视觉特征增强, 双向交互融合

Abstract: Multimodal sentiment analysis is increasingly receiving widespread attention, with the aim of utilizing multimodal information such as text and images to achieve emotion prediction. Compared to text, the visual modality, as an auxiliary modality, may contain more redundant or confounding information unrelated to emotions, and existing research does not fully consider the interaction and complementarity between multiple perceptual modalities. To address these issues, an imagetext emotion classification model based on visual feature enhancement and bidirectional interactive fusion (VFEBIF) is proposed. In this approach, the fine-grained visual feature enhancement module utilizes structured knowledge from scene graphs and filtering techniques based on CLIP to extract keywords from the text related to visual semantics, thereby enhancing local visual features. Additionally, the bidirectional interactive fusion module implements inter-modal interaction in parallel, and fuses multimodal features to thoroughly explore complementary information between text and image, thus achieving emotion classification. Experiments on two public datasets, TumEmo and MVSA-Single, demonstrate that the VFEBIF method outperforms most existing approaches and can effectively improve the performance of sentiment classification.

Key words: multimodal sentiment analysis, imagetext emotion classification, visual feature enhancement, bidirectional interactive fusion