• 中国计算机学会会刊
  • 中国科技核心期刊
  • 中文核心期刊

计算机工程与科学 ›› 2025, Vol. 47 ›› Issue (7): 1321-1330.

• 人工智能与数据挖掘 • 上一篇    

基于双通道图卷积网络的多模态方面级情感分析

张凤1,邵玉斌1,杜庆治1,龙华1,马迪南2   

  1. (1.昆明理工大学信息工程与自动化学院,云南 昆明 650504;
    2.云南省媒体融合重点实验室,云南 昆明 650228)

  • 收稿日期:2023-12-15 修回日期:2024-07-17 出版日期:2025-07-25 发布日期:2025-08-25
  • 基金资助:
    云南省媒体融合重点实验室课题(220235205)

Multimodal aspect-based sentiment analysis based on dual channel graph convolutional network

ZHANG Feng1,SHAO Yubin1,DU Qingzhi1,LONG Hua1,MA Dinan2#br#   

  1. (1.Faculty of lnformation Engineering and Automation,Kunming University of Science and Technology,Kunming 650504;
    2.Yunnan Key Laboratory of Media Convergence,Kunming 650228,China)
  • Received:2023-12-15 Revised:2024-07-17 Online:2025-07-25 Published:2025-08-25

摘要: 针对在多模态方面级情感分析任务中,传统方法主要关注图文模态交互的深层信息而较少关注图像和文本中与方面相关的浅层信息,导致引入与方面无关的噪声,使得模型在捕获方面与情感之间复杂关系的能力上受到限制的问题,提出一种双通道图卷积网络模型DCGCN。在BART模型的结构上,利用注意力机制增强方面语义,通过图卷积网络获取方面增强的多模态特征,并将句法依赖、基于方面的位置依赖和方面增强的图文相关性信息聚合到GCN邻接权重矩阵中以获得感知多信息的多模态特征。实验表明,所提DCGCN模型在Twitter的2个数据集上的F1值分别达到了67.4%和67.9%,提高了多模态方面级情感分析的性能。

关键词: 方面级情感分析, 多模态, 图卷积网络, 句法依赖, 注意力机制

Abstract: In the task of multimodal aspect-based sentiment analysis,traditional methods primarily focus on deep-level cross-modal interactions between images and texts while paying less attention to the aspect-related shallow information within images and texts.This oversight leads to the introduction of aspect-irrelevant noise,thereby limiting the model’s ability to capture the complex relationships between aspects and sentiments.To address this issue,a dual-channel graph convolutional network (DCGCN) model is proposed.Based on the architecture of the BART model,the proposed approach employs an attention mechanism to enhance aspect semantics,leverages graph convolutional networks (GCN) to extract aspect-enhanced multimodal features,and aggregates syntactic dependencies,aspect-based positional dependencies,and aspect-augmented imagetext correlation information into the GCN adjacency weight matrix to obtain multi-information-aware multimodal features.Experiments demonstrate that the proposed model achieves F1 scores of 67.4% and 67.9% on two Twitter datasets,respectively,and can improve the performance of multimodal aspect-based sentiment analysis.

Key words: aspect-based sentiment analysis, multimodal, graph convolutional network, syntactic dependency, attention mechanism