Abstract: In recent years, applying reinforcement learning to knowledge reasoning has shown promising performance, but it faces two key challenges: agents’ tendency to engage in aimless explorations and issues related to delayed and sparse rewards. To address these challenges, a multi-hop knowledge reasoning model based on reinforcement learning and predictive information embedding is proposed. Firstly, a predictive embedding information acquisition module is designed to incorporate the obtained predictive information into the reinforcement learning framework, resolving the issue of agents getting trapped in aimless exploration and selecting ineffective actions. Then, an action pruning mechanism combining predictive information with the Dropout concept is introduced during the traversal process to alleviate the problem of an excessively large action space. Additionally, LSTM is employed to retain the agent’s historical decision-making information, enabling the agent to select the most promising actions at each step. Finally, a new reward function  based on predictive information successfully mitigates the issues of delayed and sparse rewards. Experimental results on the WebQSP, PQL, and MetaQA datasets demonstrate that the proposed  model exhibits efficient performance in knowledge reasoning tasks and is well-suited for multi-hop question answering on knowledge graphs.

Key words: knowledge graph, reinforcement learning, knowledge reasoning