Abstract: Traditional image captioning methods do not generate explanatory description texts due to the lack of a priori knowledge of the real world, while the accuracy of the generated description texts is not high in some specialized fields. To address these problems, the military news image set captioning task is proposed, and a military news image set dataset is also constructed. The task has two key challenges: the description information is derived from the whole image set and the corresponding news articles; the semantics learned by the model is not sufficient. A military news image set captioning method based on image and text relevance and context guidance (ITRCG) is further proposed. Based on ITRCG, cross-modal information interaction is realized, the model is guided to learn more complete semantics, and named entity generation is assisted by label cleaning. Experimental validation is conducted on the constructed military news image set dataset, and the results show that ITRCG can effectively improve the quality of the description text and achieve improvements in all evaluation metrics.

Key words: image captioning, image and text relevance attention, context guidance attention, image set, news text