Abstract: Sequence labeling is a basic task in the field of natural language processing. At present, most sequence labeling methods use recurrent neural networks and their variants to directly extract the contextual semantic information in the sequence. Although they can effectively capture continuous dependencies between words and achieve good performance, they are not capable of capturing discrete dependencies in sequences, and also ignores the relationship between words and labels. Therefore, this paper proposes a multi-level semantic information fusion coding method. Firstly, the sequence context semantic information is extracted through the bidirectional long and short-term memory network. Secondly, the label semantic information is added to the context semantic information using the attention mechanism to obtain the new semantic information. Thirdly, a self-attention mechanism is used to capture discrete dependencies in the sequence, and obtain contextual semantic information containing discrete dependencies. Finally, a fusion mechanism is used to fuse the three types of semantic information to obtain a brand-new semantic information. The results prove that the proposed multi-level semantic information fusion encoding method significantly improves the model performance in comparison to the method of directly encoding the sequence using the recurrent neural network or its variants.

Key words: sequence labeling, multi-level semantic information fusion coding, label semantic information, attention mechanism, fusion mechanism