关键词: 虚假信息检测, 多模态, 后期融合, 多层CNN；多分类器

Abstract: Aiming at the problem that the existing multi-modal false information detection methods rarely fuse multi-modal features at the feature level and ignore the late fusion effect of multi-modal features, a false information detection method based on CNN multi-modal feature fusion and multi- classifier hybrid prediction is proposed. This method applies multi-layer CNN to multi-modal feature fusion for the first time. The model first uses BERT and Swin-transformer to extract text and image features, and then uses multi-layer CNN to fuse multi-modal features at the feature level. Modal features are fused at the sentence level. Finally, the two fusion features are input into different classifiers to obtain two probability distributions, and the two probability distributions are added proportionally to obtain the final prediction result. Compared with the attention-based multi-modal factorization bilinear model (AMFB), the accuracy of this model is improved by 6.1% and 4.3% on the Weibo dataset and Twitter dataset, respectively. The experimental results show that the proposed model can effectively improve the accuracy of false information detection. 

Key words: false information detection, multi-modal, late fusion, multi-layer CNN, multi-classifier