关键词: 常识问答, 知识注入, 预训练语言模型, 图神经网络, 注意力机制

Abstract: Commonsense Question Answering is dedicated to having models answer questions that require human commonsense knowledge. A category of methods for this task is to retrieve relevant knowledge to assist the model in answering commonsense questions. This category of methods are mainly divided into two steps: knowledge retrieval and knowledge inference. Knowledge retrieval refers to retrieving the knowledge associated with question, while knowledge inference refers to using the retrieved knowledge to answer commonsense questions. In this regard, one of the challenges facing commonsense question answering is how to find appropriate external knowledge to help answer the question. Many existing commonsense question answering models usually rely on single source of external knowledge, but it is difficult for a single source of knowledge to comprehensively cover all the required knowledge. To address this problem, this paper proposes a commonsense question answering method based on multi-source knowledge infusion. Firstly, in order to cope with the knowledge coverage problem during knowledge retrieval,  using pretrained language models to integrate knowledge from multiple sources (including structured and unstructured knowledge) to form a unified knowledge representation. Secondly, in order to make full use of the semantic relations embedded in structured knowledge during knowledge inference, model identify entity concepts and relationship paths between entities in the context to construct the entity relationship graph, and then use graph attention network to model the entity relationship graph. Finally,  using the evidence information in the entity relationships graph and entity knowledge representations to reason and answer the questions. The experimental results on the CommonsenseQA dataset show that the accuracy of the commonsense question answering method based on multi-source knowledge infusion is 79.20% and 75.02% on the verification set and test set, respectively, which exceeds the best baseline models. This verifies the effectiveness of multi-source knowledge infusion method in commonsense question answering tasks.

Key words: commonsense question answering, knowledge infusion, pre-trained language model, graph neural network, attention mechanism