Abstract: For patent text classification, the existing static word vector tools such as word2vec cannot express the context information of words, and most of the models can not completely extract features. Aiming at this problem, a multi-scale semantic collaborative patent text classification model based on RoBERTa, named RoBERTa-MCNN-BiSRU++-AT, is proposed. RoBERTa can learn the context-appropriate dynamic semantic representation of the current word and solve the problem that static word vectors cannot represent polysemous words. The multi-scale semantic collaboration model uses the convolution layer to capture the multi-scale local semantic features of text, and then uses the bidirectional built-in simple attention loop unit to model the context semantics at different levels. The multi-scale output features are spliced, and the key features that contribute more to the classification result are assigned higher weight by the attention mechanism. Experiments were carried out on the patent text data set published by the National Information Center. The results show that, compared with ALBERT-BiGRU and BiLSTM-ATT-CNN, RoBERTa-MCNN-BiSRU++-AT increases the accuracy by 2.7% and 5.1% respectively in patent text classification at the department level, and by 6.7% and 8.4% respectively in patent text classification at the major class level. RoBERTa-MCNN-BiSRU++-AT can effectively improve the classification effect of different levels of patent texts.

Key words: patent text classification, semantic collaboration, simple recurrent unit, RoBERTa model