关键词: 中文词 词 聚类 词的临近空间 EM算法

Abstract:

Word clustering is an important fundamental work of automatic language process. Point to dearth of training data and low quality of training data, whi  ch is the main obstacle of Chinese word clustering, a Chinese oriented algorithm is presented in this paper. First, the context similar degree of a word  is used as the distance measure of the word; second, the limitation of taking the distance measure only into account is analyzed; then, the concept of  Word-Near-Space is put for- ward, which can make word clustering work without allocating the total class number. Finally, according to the class which i s the result of clustering,we calculate the context similar degree, and repeat the above steps until the whole algorithm con- verges, so that it is cons istent with the EM criteriom Experiments show that the algorithm effectively conquers the two main obstacles of Chinese word clustering, and brings abou  t good clustering results.

Key words: Chinese word, clustering, Word-Near-Space, EM algorithm