Abstract:

This paper analyzes several traditional methods for the Chinese word segmentation, compares the advantages and disadvantages of these methods, and presents a new segmentation algorithm. The method adopts the improved bidirectional Markov chain statistical method to update the word library, and then uses the Reverse Maximum Match method based on the word library and the GameTree search algorithm to cut the Chinese word strings. The experimental results show this algorithm has got better effect on veracity, efficiency and new word distinguishment.

Key words: forward maximum match, reverse maximum match, statistical method, definite finite automation