基于资源分配网络和语义特征选取的文本分类

J4 ›› 2014, Vol. 36 ›› Issue (02): 340-346.

基于资源分配网络和语义特征选取的文本分类

何晓亮1,2，宋威1，梁久祯1

(1.江南大学物联网工程学院，江苏无锡 214122;2.公安部交通管理科学研究所，江苏无锡 214151）

收稿日期:2012-08-13 修回日期:2012-10-08 出版日期:2014-02-25 发布日期:2014-02-25
基金资助:
国家自然科学青年基金资助项目(61103129);博士点新教师专项研究基金资助项目(20100093120004);中央高校基本科研业务费专项资金资助项目(JUSRP11130);江苏省自然科学基金资助项目(SBK201122266)

Text categorization based on resource allocating network and semantic feature selection

HE Xiaoliang1,2，SONG Wei1，LIANG Jiuzhen1

1.School of IoT Engineering,Jiangnan University,Wuxi 214122;
2.Traffic Management Research Institute,Ministry of Public Security,Wuxi 214151,China)

Received:2012-08-13 Revised:2012-10-08 Online:2014-02-25 Published:2014-02-25

摘要/Abstract

摘要：

针对资源分配网络(RAN)算法存在隐含层节点受初始学习数据影响大、收敛速度低等问题，提出一种新的RAN学习算法。通过均值算法确定初始隐含层节点，在原有的“新颖性准则”基础上增加RMS窗口，更好地判定隐含层节点是否增加。同时，采用最小均方(LMS)算法与扩展卡尔曼滤波器(EKF)算法相结合调整网络参数，提高算法学习速度。由于基于词向量空间文本模型很难处理文本的高维特性和语义复杂性，为此通过语义特征选取方法对文本输入空间进行语义特征的抽取和降维。实验结果表明，新的RAN学习算法具有学习速度快、网络结构紧凑、分类效果好的优点，而且，在语义特征选取的同时实现了降维，大幅度减少文本分类时间，有效提高了系统分类准确性。

关键词: RAN学习算法, 径向基函数, 语义特征选取, 扩展卡尔曼滤波器算法, 最小均方算法, 文本分类

Abstract:

Confronted with the existence of hidden nodes affected by the initial learning data and the low convergence rate of RAN learning algorithm, a new Resource Allocating Network (RAN) learning algorithm is proposed. The initial hidden layer node, determined through Kmeans algorithm, adding the 'RMS window’ based on the novelty rule, can better judge whether to increase hidden layer nodes or not. Meanwhile, the network parameters are adjusted by combining Least Mean Squares algorithm and Extended Kalman Filter algorithm, thus improving the learning rate. Since it is rather difficult to deal with the high dimension characteristics and complex semantic character of texts through words space text categorization method, we reduce the dimension and extract the semantic character space to the text input space through the semantic feature selection method. The experimental results show that the new RAN algorithm has the advantage of highspeed learning, compact network structure and good classification. Moreover, semantic feature selection can not only achieve the reduction of dimension and categorization time, but also raise the accuracy of the categorizing system effectively.

Key words: RAN learning algorithm;radial basis function;semantic feature selection;extended Kalman filter algorithm;least mean squares algorithm;text categorization

何晓亮1,2，宋威1，梁久祯1. 基于资源分配网络和语义特征选取的文本分类[J]. J4, 2014, 36(02): 340-346.

HE Xiaoliang1,2，SONG Wei1，LIANG Jiuzhen1. Text categorization based on resource allocating network and semantic feature selection [J]. J4, 2014, 36(02): 340-346.

参考文献

［1］Lin H T, Lin J C, Weng R C. A note on platt’s probabilistic outputs for support vector machines［J］. Machine Learning, 2007, 68(10):267276.
［2］Plakua E K, Avraki L. Distributed computation of the KNN graph for large highdimensional point sets［J］. Journal of Parallel and Distributed Computing, 2007, 67(3):346359.
［3］Guo Zhaohui,Liu Shaohan,Wu Gangshan. Feature selection for neural networkbased Chinese text categorization［J］.Application Research of Computers, 2006,23(7):161164.(in Chinese)
［4］Chen Jingnian, Huang Houkuan, Tian Fengzhan, et al.Method of feature selection for text categorization with Bayesian classifiers［J］.Computer Engineering and Application,2008,44(13):2427.(in Chinese)
［5］Wang Yu,Wang Zhengou. Text categorization rule extraction based on fuzzy decision tree［J］.Computer Applications,2005,25(7):16341637.(in Chinese)
［6］Mao J, Jain K. Artificial neural networks for feature extraction and multivariate data projection［J］. IEEE Transactions on Neural Networks, 1995, 6(2):296317.
［7］Song H H, Lee S W. A selforganizing neural Ttree for largeset pattern classification［J］. IEEE Transactions on Neural Networks, 1998, 9(5):369380.
［8］Yuan J L, Fine T L. Neuralnetwork design for small training sets of high dimension［J］. IEEE Transactions on Neural Networks, 1998, 9(1):266280.
［9］Mukhopadhyay S, Roy A, Kim L S. A polynomial time algorithm for generating neural networks for pattern classification:Its stability properties and some test results［J］. Neural Computation, 1993, 5(2):317330.
［10］Chen Q Y. Generatingshrinking algorithm for learning arbitrary classification［J］. Neural Networks, 1994, 7(9):14771489.
［11］Poggio T, Girosi F. Networks for approximation and learning［J］. Proceedings of the IEEE, 1990, 78(9):14811497.
［12］Parekh R, Yang J. Constructive neuralnetwork learning algorithms for pattern classification［J］. IEEE Transactions on Neural Networks, 2000, 11(2):436451.
［13］Platt J. A resource allocating network for function interpolation［J］. Neural Computation, 1991, 3(2):213225.
［14］Manolis W, Nicolas T, Stefanos K. Intelligent initialization of resource allocating RBF networks［J］. Neural Networks, 2005, 18(2):117122.
［15］Kadirkamanathan V,Niranjan M.A function estimation approach to sequential learning with neural networks［J］. Neural Computation, 1993, 5(6):954975.
［16］Li Bin.An Improvement of the RAN learning algorithm［J］.Pattern Recognition and Artificial Intelligence,2006,19(2):220226.(in Chinese)
［17］Su Jinshu, Zhang Bofeng, Xu Xin.Advances in machine learning based text categorization［J］.Journal of Software, 2006,17(9):18481859.(in Chinese)
［18］Song Wei , Wang Shitong, Li Chenghua. Parametric and nonparametric evolutionary computing with a contentbased feature selection approach for parallel categorization［J］. Expert System with Application, 2009, 36(9):737743.
［19］Deerwester S C, Dumais S T, Landauer T K, et al. Indexing by latent semantic analysis［J］. Journal of the American Society of Information Science, 1990, 41(6):391–407.
［20］Li C H, Park S C. Combination of modified BPNN algorithms and an efficient feature selection method for text categorization［J］. Information Processing and Management, 2009, 45(3):329340.
附中文参考文献：
［3］郭昭辉,刘绍翰,武港山.基于神经网络的中文文本分类中的特征选择技术［J］.计算机应用研究,2006,23(7):161164.
［4］陈景年,黄厚宽,田凤占，等.一种用于贝叶斯分类器的文本特征选择方法［J］.计算机工程与应用, 2008, 44(13):2427.
［5］王煜,王正欧.基于模糊决策树的文本分类规则抽取［J］.计算机应用,2005,25(7):16341637.
［16］李彬.一种改进的RAN学习算法［J］.模式识别与人工智能,2006,19(2):220226.
［17］苏金树，张博锋，徐昕.基于机器学习的文本分类技术研究进展［J］.软件学报, 2006,17(9):18481859.

[1]	许浪, 李代伟, 张海清, 唐聃, 何磊, 于曦. 基于神经网络的医疗文本分类研究[J]. 计算机工程与科学, 2023, 45(06): 1116-1122.
[2]	梅侠峰, 吴晓鸰, 黄泽民, 凌捷. 融合RoBERTa的多尺度语义协同专利文本分类模型[J]. 计算机工程与科学, 2023, 45(05): 903-910.
[3]	文武, 万玉辉, 张许红, 文志云, . 基于改进CHI和PCA的文本特征选择[J]. 计算机工程与科学, 2021, 43(09): 1645-1652.
[4]	张丽, 马静. 融合词语统计特征和语义信息的文本分类方法研究[J]. 计算机工程与科学, 2021, 43(07): 1308-1315.
[5]	李晓红, 王闪闪, 马堉银, 马慧芳. 融合相似度图和随机游走模型的多标签短文本分类算法[J]. 计算机工程与科学, 2021, 43(06): 1081-1087.
[6]	李校林, 王成, . 一种基于质心的多标签文本分类模型研究[J]. 计算机工程与科学, 2020, 42(06): 1120-1126.
[7]	葛晓伟, 李凯霞, 程铭. 基于CNN-SVM的护理不良事件文本分类研究[J]. 计算机工程与科学, 2020, 42(01): 161-166.
[8]	黄贤英,熊李媛,刘英涛,李沁东. 基于类别特征改进的KNN短文本分类算法[J]. 计算机工程与科学, 2018, 40(01): 148-154.
[9]	刘述昌,张忠林. 基于中心向量的多级分类KNN算法研究[J]. 计算机工程与科学, 2017, 39(09): 1758-1764.
[10]	马长林，杨正良，谢罗迪. 文本分类中CTM模型的优化和可视化应用研究[J]. 计算机工程与科学, 2017, 39(03): 599-604.
[11]	马慧芳，周汝南，吉余岗，鲁小勇. 融合词语类别特征和语义的短文本分类方法[J]. 计算机工程与科学, 2017, 39(02): 399-404.
[12]	阿丽亚·艾尔肯，哈力旦·阿布都热依木，何燕，吴冰冰. 基于深度置信网络的维吾尔文垃圾短信分类技术研究[J]. 计算机工程与科学, 2016, 38(10): 2134-2139.
[13]	周杭霞，叶佳骏，任欢. 基于快速自编码的RELM的文本分类[J]. J4, 2016, 38(05): 871-876.
[14]	谭光兴，刘臻晖. 基于SVM的局部潜在语义分析算法研究[J]. J4, 2016, 38(01): 177-182.
[15]	戴上平1，冯鹏1，刘盛英杰1，舒红2. 基于余弦距离的局部敏感哈希的KNN算法在中文文本上的快速分类[J]. J4, 2015, 37(10): 1971-1976.