J4 ›› 2008, Vol. 30 ›› Issue (8): 79-82.
• 论文 • 上一篇 下一篇
吴涛 张毛迪 陈传波
出版日期:
发布日期:
Online:
Published:
摘要：
在比较各种传统分词方法优缺点的基础上，本文提出了一种新的分词算法。它采用改进的双向Markov链统计方法对词库进行更新，再利用基于词典的有穷自动机后串最大匹配算法以及博弈树搜索算法进行分词。实验结果表明，该分词算法在分词准确性、效率以及生词辨识上取得了良好的效果。
关键词: 正向最大前串匹配 逆向最大前串匹配 统计法 有穷自动机
Abstract:
This paper analyzes several traditional methods for the Chinese word segmentation, compares the advantages and disadvantages of these methods, and presents a new segmentation algorithm. The method adopts the improved bidirectional Markov chain statistical method to update the word library, and then uses the Reverse Maximum Match method based on the word library and the GameTree search algorithm to cut the Chinese word strings. The experimental results show this algorithm has got better effect on veracity, efficiency and new word distinguishment.
Key words: forward maximum match, reverse maximum match, statistical method, definite finite automation
吴涛 张毛迪 陈传波. 一种改进的统计与后串最大匹配的中文分词算法研究[J]. J4, 2008, 30(8): 79-82.
0 / / 推荐
导出引用管理器 EndNote|Ris|BibTeX
链接本文: http://joces.nudt.edu.cn/CN/
http://joces.nudt.edu.cn/CN/Y2008/V30/I8/79
