基于影响度剪枝的极端学习机分类算法研究

J4 ›› 2016, Vol. 38 ›› Issue (04): 699-705.

基于影响度剪枝的极端学习机分类算法研究

张辉1,2，师统1，王耀南2

（1.长沙理工大学电气与信息工程学院，湖南长沙 410012；
2.湖南大学电气与信息工程学院，湖南长沙 410082）

收稿日期:2015-08-25 修回日期:2015-10-20 出版日期:2016-04-25 发布日期:2016-04-25
基金资助:
国家自然科学基金（61401046）;国家科技支撑计划(2015BAF11B01);湖南省自然科学基金(13JJ4058);湖南省教育厅科学研究青年项目（13B135）;图像测量与视觉导航湖南省重点实验室开放课题（TXCLKF2013001）;长沙市科技计划项目（K140401911）

A classification algorithm of extreme learning
machine based on influence degree pruning

ZHANG Hui1,2，SHI Tong1，WANG Yaonan2

（1.College of Electrical and Information Engineering,Changsha University of Science and Technology,Changsha 410012；
2.College of Electrical and Information Engineering,Hunan University,Changsha 410082,China）

Received:2015-08-25 Revised:2015-10-20 Online:2016-04-25 Published:2016-04-25

摘要/Abstract

摘要：

针对极端学习机（ELM）网络规模控制问题，从剪枝思路出发，提出了一种基于影响度剪枝的ELM分类算法。利用ELM网络单个隐节点连接输入层和输出层的权值向量、该隐节点的输出、初始隐节点个数以及训练样本个数，定义单个隐节点相对于整个网络学习的影响度，根据影响度判断隐节点的重要性并将其排序，采用与ELM网络规模相匹配的剪枝步长删除冗余节点，最后更新隐含层与输入层和输出层连接的权值向量。通过对多个UCI机器学习数据集进行分类实验，并将提出的算法与EMELM、PELM和ELM算法相比较，结果表明，该算法具有较高的稳定性和测试精度，训练速度较快，并能有效地控制网络规模。

关键词: 极端学习机, 影响度分析, 剪枝算法, 网络规模, 分类算法

Abstract:

To slove the network size control problems of the extremely learning machine (ELM), we propose an ELM classification algorithm based on the influence degree pruning. The algorithm uses the individual ELM hidden node which connects the input and output layer weight vector, the output of the node, the number of samples and the initial number of hidden nodes to define the influence degree of the hidden node on the entire network. Then the importance of the hidden node is determined by the sorted influence degree, and the pruning step length which matches the ELM network scale is used to delete redundant nodes. Finally the weight vectors are updated. We categorize several practical problems on UCI data sets through experiments, and compare the proposed algorithm with the EMELM, PELM and ELM. Experimental results show that the proposed algorithm has higher stability and precision and faster training speed, and it can control the network size effectively.

Key words: ELM;analysis of influence degree;pruning algorithm;network size;classification algorithm

张辉1,2，师统1，王耀南2. 基于影响度剪枝的极端学习机分类算法研究[J]. J4, 2016, 38(04): 699-705.

ZHANG Hui1,2，SHI Tong1，WANG Yaonan2. A classification algorithm of extreme learning
machine based on influence degree pruning [J]. J4, 2016, 38(04): 699-705.

参考文献［15］

［1］	Huang G B,Zhu Q Y,Siew C K. Extreme learning machine:Theory and applications［J］. Neurocomputing,2006,70(1/2/3):489501.
［2］	Huang G B,Chen L. Enhanced random search based incremental extreme learning machine［J］. Neurocomputing,2008,71 (1618):34603468.
［3］	Wang Guimei, Jiang Chao. Coal mining machine fault diagnosis method based on ELM neural network［J］. Coal Mine Machinery,2014,35(1):239240.(in Chinese)
［4］	Decherchi S. CircularELM for the reduced reference assessment of perceived image quality［J］. Neurocomputing,2013,102(1):7889.
［5］	Chen H, Zhang J, Zhang X G, et al. A robustELM approach based on parzen windiow’s estimation for kiln sintering temperature detection［J］.Acta Automatic Sinica, 2012, 38(5):841849. (in Chinese)
［6］	Huang G B,Wang D H. Extreme learning machines:A survey［J］. International Journal of Machine Learning and Cybernetics,2011,2(2):107122.
［7］	Wei Yanling. Research on application of ELM in the prediction of insect pest［J］.Science Technology and Engineering,2012,12(22):55805583. (in Chinese)
［8］	Zhai Min, Zhang Rui. Extreme learning machine (ELM) survey of network structure adjustment method［J］. Journal of Xi'an University of Arts and Science(Natural Science Edition),2014,17(1):16. (in Chinese)
［9］	Huang G B,Chen L. Universal approximate using incremental constructive feed forward networks with random hidden nodes［J］. IEEE Transactions on Neural Network,2006,17(4):879892.
［10］	Huang G B,Chen L. Convex incremental extreme learning machine［J］. Neurocomputing,2007,70(1618):30563062.
［11］	Han M, Li D C. An normal regularization term ELM algorithm based on surrogate function and Bayesian framework［J］.Acta Automatic Sinica, 2011,37(11): 13441350. (in Chinese)
［12］	Boyd S,Chaoul L E. Linear matrix inequalities in system and control theory［M］. Philadelphia,PA:SIAM,1994．
［13］	Miche Y, Sorjamaa A, Bas P, et al. OPELM:Optimally pruned extreme learning machine［J］. IEEE Transactions on Neural Networks, 2010,21(1):158162.
［14］	Li F J, Han H G, Qiao J F. ELM pruning algorithm based on sensitivity analysis［J］.Control and Decision, 2014,29 (4):16. (in Chinese)
［15］	Zhang Z Z, Qiao J F, Han H G. A pruning algorithm based on neural complexity［J］.Control and Decision, 2010,25(6):821830. (in Chinese)
	附中文参考文献：
［3］	王桂梅,蒋超. 基于ELM 神经网络的采煤机故障诊断方法［J］. 煤矿机械,2014,35(1):239240.
［5］	陈华,章兢,张小刚. 一种基于Parzen 窗估计的鲁棒ELM烧结温度检测方法［J］. 自动化学报,2012,38(5):841849.
［7］	韦艳玲.ELM 在虫害预测中的应用研究［J］. 科学技术与工程,2012,12(22):55805583.
［8］	翟敏,张瑞. 极限学习机ELM网络结构调整方法综述［J］. 西安文理学院学报（自然科学版）,2014,17(1):16.
［11］	韩敏,李德才. 基于替代函数及贝叶斯框架的1 范数ELM算法［J］. 自动化学报,2011,37(11):13441350.
［14］	李凡军,韩红桂,乔俊飞. 基于灵敏度分析法的ELM剪枝算法［J］. 控制与决策,2014,29(4):16.
［15］	张昭昭,乔俊飞,韩红桂.一种基于神经网络复杂度的修剪算法［J］. 控制与决策,2010,25(6):821830.

[1]	周美琴，陈诗旭，袁鼎荣，朱新华. 一种单位代价收益决策树剪枝算法[J]. J4, 2016, 38(05): 1023-1030.
[2]	黄贤英,陈红阳,刘英涛,熊李媛. 一种新的微博短文本特征词选择算法[J]. J4, 2015, 37(09): 1761-1767.
[3]	郑重,王意洁,马行空. SubCounter:一种基于语义簇聚的节点子集规模估计方法[J]. J4, 2012, 34(2): 104-110.
[4]	严胜样吴绍春吴耿锋金沈杰. 一种基于纵向划分数据集的并行决策树分类算法[J]. J4, 2004, 26(7): 67-70.