• 中国计算机学会会刊
  • 中国科技核心期刊
  • 中文核心期刊

Computer Engineering & Science ›› 2026, Vol. 48 ›› Issue (2): 372-380.

• Artificial Intelligence and Data Mining • Previous Articles    

Adaptive fusion for multimodal entity alignment method

WANG Yiyan,WANG Hairong,WANG Yimeng,WANG Wenlong#br#

#br#
  

  1. (1.School of Computer Science and Engineering,North Minzu University,Yinchuan 750021;
    2.School of Computer Science,Zhuhai College of Science and Technology,Zhuhai 519041,China)
  • Received:2024-04-08 Revised:2024-09-26 Online:2026-02-25 Published:2026-03-10

Abstract: To address the issues of information loss during feature fusion and incorrect entity alignment caused by solely focusing on joint entity vectors in multimodal entity alignment, this paper proposes an adaptive fusion  for multimodal entity alignment method(ADMMEA). This method employs FastText, ResNet-152, and GAT models to extract multimodal entity features, obtaining feature representations for entity names, images, and structural data. It utilizes the Bray-Curtis dissimilarity matrix and Levenshtein distance to calculate the similarity between source and target entities, generating distance matrices for each modality. Through an adaptive fusion strategy, the text-image distance matrices are fused and then concatenated with the structural information matrix to obtain the final fused matrix. Leveraging a ranking approach, the fused matrix is sorted in descending order based on similarity scores to achieve multimodal entity alignment. Experimental evaluations are conducted on the ZH-EN, JA-EN, and FR-EN subsets of the DBP15K dataset, and the results are compared with 13 other methods, including JAPE, RDGCN, MOGNN, and MIMEA, etc. The findings demonstrate that ADMMEA achieves Hits@1 scores of 0.985, 0.995, and 0.994 on the ZH-EN,JA-EN and FR-EN datasets, respectively, validating the effectiveness of the ADMMEA.

Key words: multimodal knowledge graphs, multimodal entity alignment, embedding model, adaptive fusion, matching problem