基于局部收敛权阵进化的BP神经网络MapReduce训练

计算机工程与科学

基于局部收敛权阵进化的BP神经网络MapReduce训练

陈旺虎，俞茂义，马生俊，李金溶，郏文博

（西北师范大学计算机科学与工程学院，甘肃兰州 730070）

收稿日期:2016-08-26 修回日期:2016-10-20 出版日期:2016-12-25 发布日期:2016-12-25
基金资助:
国家自然科学基金（61462076）；甘肃省自然科学基金（1104GKCA023）；甘肃省科技攻关项目（1208RJZA134）；西北师范大学青年教师科研提升计划（NWNULKQN1230）

MapReduce training of BP neural networks

based on local weight matrix evolution

CHEN Wanghu,YU Maoyi,MA Shengjun,LI Jingrong,JIA Wenbo

(College of Computer Science and Engineering,Northwest Normal University,Lanzhou 730070,China)

Received:2016-08-26 Revised:2016-10-20 Online:2016-12-25 Published:2016-12-25

摘要/Abstract

摘要：

为提高大样本集情况下BP神经网络的训练效率，提出了一种基于局部收敛权阵进化的BP神经网络MapReduce训练方法，以各Map任务基于其输入数据分片训练产生的局部收敛权阵作为初始种群，在Reduce任务中通过种群进化，选取适应度最高的权阵作为Map任务下一轮训练的初始权阵，直至该权阵对所有输入数据分片收敛。实验结果表明，与现有方法相比，该方法可有效避免MapReduce训练BP神经网络时容易陷入局部收敛的问题，并大大减少训练时间。

关键词: BP神经网络, 遗传算法, MapReduce, 收敛

Abstract:

To improve the efficiency of BP neural network (BPNN) training, we propose an approach with MapReduce based on the local weight matrix evolution. The local weight matrices produced by Map tasks based on its input data splits are passed to Reduce tasks as the initial population of a genetic algorithm. After the evolution of the current population, the weight matrix with highest fitness is selected as the initial weight matrix in the next turn of training. The training does not stop until the weight matrix is convergent on the whole sample data. Experiments show that the proposed approach can maintain the global convergence of a BPNN on its whole training sample data, and improve the efficiency of BP-NN training with MapReduce.

Key words: BP neural network, genetic algorithm, MapReduce, convergence

陈旺虎，俞茂义，马生俊，李金溶，郏文博. 基于局部收敛权阵进化的BP神经网络MapReduce训练[J]. 计算机工程与科学.

CHEN Wanghu,YU Maoyi,MA Shengjun,LI Jingrong,JIA Wenbo.

MapReduce training of BP neural networks

based on local weight matrix evolution

[J]. Computer Engineering & Science.

参考文献［13］

［1］	Huang Yihfang.Artificial neural networkslearning and generalization［C］∥Proc of IEEE AsiaPacific Conference on Circuits and Systems,1994:162172.
［2］	Xu Zhonghua,Yang Weifeng,Jiang Weijin,et al.Improvement and application of the learning algorithm of BP neural networks［J］.Computer Engineering & Science,2004,26(12):6163.(in Chinese)
［3］	Li Boyang,Liu Chen.Parallel BP neural network on singlechip cloud computer［C］∥Proc of IEEE International Conference on High Performance Computing and Communications,2015:18711875.
［4］	Fedorova N, Terekhoff S. Parallel MPI implementation of training algorithms for mediumsize feedforward neural networks［C］∥Proc of International Joint Conference on Neural Networks,1999:23782379.
［5］	Joldos M, Vinteler O, PeterI R, et al. MPIbased asynchronous simulation of spiking neural networks on the grid［C］∥Proc of International Symposium on Symbolic and Numeric Algorithms for Scientific Computing,2013:481487.
［6］	Liu Zhiqiang, Li Hongyan, Miao Gaoshan. MapReducebased backpropagation neural network over large scale mobile data［C］∥Proc of IEEE International Conference on Natural Computation,2010:17261730.
［7］	Zhang Rui, Jiang Changbing. The bank risk forewarning model of BP neural network based on the cloud computing［C］∥Proc of IEEE International Conference on Computing and Networking Technology,2012:9194.
［8］	Jia Yusheng,Zhu Qing.A method for text categorization using BP network based on Hadoop［C］∥Proc of IEEE International Conference on Computational and Information Sciences,2013:818821.
［9］	Zhou Binhan, Wang Wenjun, Zhang Xiangfeng.Training backpropagation neural network in MapReduce［C］∥Proc of International Conference on Computer,Communications and Infomation Technology,2014:2225.
［10］	Maulik U,Bandyopadhyay S.Genetic algorithmbased clustering technique［J］.Pattern Recognition,2000,33(9):14551465.
［11］	Ding Shifei,Su Chunyang,Yu Junzhao.An optimizing BP neural network algorithm based on genetic algorithm［J］.Artificial Intelligence Review,2011,36(2):153162.
［12］	Zhu Chenje, Rao Ruonan. The improved BP algorithm based on MapReduce and genetic algorithm［C］∥Proc of International Conference on Computer Science and Service System,2012:15671570.
［13］	Tian Xuguang, Song Tong, Liu Yuxin. Optimizing the structure and parameters of BP neural network based on genetic algorithm［J］.Computer Applications & Software,2004,21(6):6971.(in Chinese)
［14］	Gu Rong,Shen Furao,Huang Yihua.A parallel computing platform for training large scale neural networks［C］∥Proc of IEEE International Conference on Big Data,2013:376384.
［15］	Liu Yang,Jing Weizhe,Xu Lixiong.Parallelizing backpropagation neural network using MapReduce and cascading model［J］.Computational Intelligence and Neuroscience,2016,2016(2):111.
［16］	Liu Yang,Xu Lixiong,Li Maozhen.The parallelization of back propagation neural network in MapReduce and spark［J］.International Journal of Parallel Programming,2016,44:120.
	附中文参考文献：
［2］	许中华，杨伟丰，蒋伟进,等.BP神经网络学习算法的改进与应用［J］.计算机工程与科学,2004,26(12):6163.
［13］	田旭光,宋彤,刘宇新.结合遗传算法优化BP神经网络的结构和参数［J］.计算机应用与软件,2004,21(6):6971.

[1]	杨炳媛, 袁杰, 郭园园. 一种自适应鲸鱼快速优化算法[J]. 计算机工程与科学, 2023, 45(01): 145-153.
[2]	吴思婷, 鲍亮. 非Hermitian正定线性方程组的外推的广义HSS方法[J]. 计算机工程与科学, 2022, 44(10): 1885-1892.
[3]	耿召里, 李目, 曹淑睿, 刘昶忻. 基于混合反向学习策略的鲸鱼优化算法[J]. 计算机工程与科学, 2022, 44(02): 355-363.
[4]	毛媛媛, 石恩雅, 蒋从锋, 仇烨亮, 贾刚勇, 万健, 闫龙川. 基于遗传算法的数据中心能效仿真[J]. 计算机工程与科学, 2021, 43(08): 1341-1352.
[5]	胡新荣, 刘嘉文, 刘军平, 彭涛, 何儒汉, . 基于改进GA-BP-MC神经网络的人体三维尺寸预测[J]. 计算机工程与科学, 2021, 43(08): 1443-1453.
[6]	李静, 莫思敏 . 基于改进遗传算法的深度神经网络优化研究[J]. 计算机工程与科学, 2021, 43(08): 1503-1511.
[7]	杨坚伟, 孟敏, 黄家乐, 武继刚. 分布式训练异构任务调度算法研究[J]. 计算机工程与科学, 2021, 43(07): 1160-1167.
[8]	王栎桥, 张达敏, 樊英, 徐航, 王依柔. 基于改进型柯西变异灰狼优化算法训练的多层感知器[J]. 计算机工程与科学, 2021, 43(06): 1131-1140.
[9]	李晶晶, 许少华. 基于组合核函数的径向基过程神经网络及其在示功图诊断中的应用[J]. 计算机工程与科学, 2021, 43(04): 746-752.
[10]	季子豪, 江凌云. 一种基于遗传算法的多站点协同计算卸载算法[J]. 计算机工程与科学, 2021, 43(03): 426-434.
[11]	邓翔宇, 张屹南, 杨雅涵. 一种用于交通标识分类的形状识别算法[J]. 计算机工程与科学, 2021, 43(02): 322-328.
[12]	贾俊杰, 秦海涛. 基于遗传算法的动态轨迹匿名算法[J]. 计算机工程与科学, 2021, 43(01): 142-150.
[13]	李丹莲, 曹倩, 徐菲. 基于改进遗传算法的连锁便利店配送路径优化[J]. 计算机工程与科学, 2020, 42(11): 2096-2102.
[14]	沈晓燕, 王雪梅, 王燕. 基于样本熵和模式识别的脑电信号识别算法研究[J]. 计算机工程与科学, 2020, 42(08): 1482-1488.
[15]	陈旺虎，田真，张礼智，梁小燕，高雅琼. 基于插值的高维稀疏数据离群点检测方法[J]. 计算机工程与科学, 2020, 42(06): 966-972.