一种基于文本相似度矩阵运算的非结构化海量投诉数据分类算法

J4 ›› 2012, Vol. 34 ›› Issue (1): 103-107.

一种基于文本相似度矩阵运算的非结构化海量投诉数据分类算法

李青1，陈阳2，谢浩然1,蒙圣光3

(1.香港城市大学计算机科学系，香港九龙塘 999077;
2.中国移动通信集团广西有限公司，广西南宁 530000;
3.珠海市发思特软件技术有限公司，广东珠海 519080)

收稿日期:2010-05-20 修回日期:2010-10-26 出版日期:2012-01-25 发布日期:2012-01-25

A Text Similarity Matrix OperationBased Classification Algorithm for Largescale Unstructured Complaint Data

LI Qing1,CHEN Yang2,XIE Haoran1,MENG Shengguang3

(1.Department of Computer Science,City University of Hong Kong, Kowloon Tong, Hong Kong SAR 999077;
2.China Mobile Corporation Guangxi Co. Ltd., Nanning 530000;
3.Faster Software Technology Co. Ltd., Zhuhai 519080,China)

Received:2010-05-20 Revised:2010-10-26 Online:2012-01-25 Published:2012-01-25

摘要/Abstract

摘要：

随着互联网和信息技术的日新月异，非结构化数据量有呈几何级数增长的趋势。尤其是Web 2.0网络社区的流行与火爆，使得增长趋势得到了进一步的加速。因此，面对海量的非结构化数据，如何有效地管理和组织它们，以便于终端用户进行信息存取，成为了一个迫在眉睫的重要研究课题。本文通过对非结构化数据的文本的建模和文本相似度比较，对于大规模非结构化数据的分类算法进行了讨论和研究，并将此算法应用到了中国移动的投诉数据分类系统中。在系统实施后，非常有效地提高了投诉数据的处理效率，从而印证所提出分类算法及系统框架的有效性。

关键词: 文本相似度, 非结构化数据, 投诉数据分类系统

Abstract:

With the fast development of the Internet and information technology nowadays, the growth of the volume of unstructured data is exponential. In particular, the prevalence of the Web 2.0 network community further enlarges the growth tendency. Therefore, how to manage and organize largescale unstructured data effectively, so as to facilitate enduser information access, becomes an urgent and important research topic. In this paper, based on the text of unstructured data modeling and text similarity, the existing largescale unstructured data classification algorithms are surveyed and discussed, and they are applied to a China Mobile user complaint data classification system. Upon the latter, the effectiveness of processing the complaint data is shown to have been much improved, and the usage of our proposed classification algorithm and system architecture is verified.

Key words: text similarity;unstructured data;complaint data classification system

李青1，陈阳2，谢浩然1,蒙圣光3. 一种基于文本相似度矩阵运算的非结构化海量投诉数据分类算法[J]. J4, 2012, 34(1): 103-107.

LI Qing1,CHEN Yang2,XIE Haoran1,MENG Shengguang3. A Text Similarity Matrix OperationBased Classification Algorithm for Largescale Unstructured Complaint Data[J]. J4, 2012, 34(1): 103-107.

参考文献［8］

［1］	http://www.facebook.com/.
［2］	http://delicious.com/.
［3］	http://twitter.com/.
［4］	Qun L. A Survey on Chinese Word Segmentation Technology［C］∥Proc of the 8th Int’l Symposuim on Nature Language Processing, 2009:4.
［5］	Yang W, Ren L, Tang R. A Dictionary Mechanism for Chinese Word Segmentation Based on the Finite Automata［C］∥Proc of 2010 Int’l Conference on Asian Language Processing, 2010:3942.
［6］	Noll M G,Meinel C. Web Search Personalization via Social Bookmarking and Tagging［C］∥Proc of the 6th Int’l the Semantic Web and the 2nd Asian Conference on Asian Semantic Web Conference, 2007:367380.
［7］	Xu S, Bao S, Fei B, et al. Exploring Folksonomy for Personalized Search［C］∥Proc of the 31st Annual Int’l ACM SIGIR Conference on Research and Development in Information Retrieval, 2008:155162.
［8］	Valluet D, Cantador I, Jose J M. Personalized Web Search with FolksonomyBased User and Document Profiles［C］∥Proc of the 32nd Annual European Conference on Information Retrieval, 2010:420431.

[1]	袁野, 廖薇. 基于多重相关信息交互的文本相似度计算方法[J]. 计算机工程与科学, 2022, 44(07): 1313-1320.
[2]	陈健鹏, 陈剑, 佘祥荣, 水新莹, 陈刚. 混合神经网络模型与注意力机制的地址匹配算法[J]. 计算机工程与科学, 2022, 44(05): 901-909.
[3]	曹小鹏, 邵一萌. ESM:一种增强语义匹配的语句评分模型[J]. 计算机工程与科学, 2020, 42(06): 1135-1140.
[4]	王君泽1,2，马洪晶1,2，张毅1,2,杨兰蓉1,2. 裁判文书类案推送中的案情相似度计算模型研究[J]. 计算机工程与科学, 2019, 41(12): 2193-2201.
[5]	程蔚1,2,线岩团1,2,周兰江1,2,余正涛1,2,王红斌1,2. 基于双语LDA的跨语言文本相似度计算方法研究[J]. 计算机工程与科学, 2017, 39(05): 978-983.
[6]	郑宗良. 基于改进DBSCAN算法的应急预案编制[J]. J4, 2012, 34(9): 149-153.
[7]	曹恬周丽张国煊. 一种基于词共现的文本相似度计算[J]. J4, 2007, 29(3): 52-53.