Abstract: In learning to rank methods, the method of learning the ranking model by directly optimizing the information retrieval evaluation indexes achieves good ranking effect, but its loss function still needs to be improved in using all ranking location information and fusing diversity ranking factors. Therefore, a diversity document ranking algorithm based on reinforcement learning is proposed. Firstly, the idea of reinforcement learning is applied to the ranking problem. By modeling the ranking behavior as a Markov decision process, the information of all ranking positions is used in each iteration to contin- uously select the optimal document for each ranking position. Secondly, the diversity strategy is used in the ranking process to cut highly similar documents according to the similarity threshold to ensure the diversity of the ranking results. Finally, the experimental results on the public dataset show that the proposed algorithm enhances the diversity of the ranking results while ensuring the ranking accuracy.

Key words: reinforcement learning, learning to rank, Markov decision process, diversity, policy gra- dient