面向磁盘故障预测的机器学习方法比较

J4 ›› 2015, Vol. 37 ›› Issue (12): 2200-2207.

面向磁盘故障预测的机器学习方法比较

董勇，蒋艳凰，卢宇彤,周恩强

(1.国防科学技术大学计算机学院,湖南长沙 410073;2. 高性能计算国家重点实验室,湖南长沙 410073)

收稿日期:2015-08-03 修回日期:2015-10-16 出版日期:2015-12-25 发布日期:2015-12-25
基金资助:
国家863计划资助项目（2012AA01A301）;国家自然科学基金资助项目（61272141,61303068,61120106005）

Comparison of machine learning methods
for disk failure prediction

DONG Yong，JIANG Yanhuang，LU Yutong，ZHOU Enqiang

(1.College of Computer,National University of Defense Technology,Changsha 410073;2.State Key Laboratory of High Performance Computing,Changsha 410073,China)

Received:2015-08-03 Revised:2015-10-16 Online:2015-12-25 Published:2015-12-25

摘要/Abstract

摘要：

磁盘是保存数据的重要载体，提高磁盘的可靠性和数据可用性具有重要意义。现代磁盘普遍支持SMART协议，用来监控磁盘的内部工作状态。采用机器学习方法，分析磁盘的SMART信息，实现对磁盘故障的预测。所采用的机器学习方法包括反向神经网络、决策树、支持向量机以及简单贝叶斯，并采用实际磁盘SMART数据进行验证与分析。基于上述数据，对不同机器学习方法的有效性及其效果进行了对比。结果表明，决策树方法的预测率最好，支持向量机方法的误报率最低。

关键词: 磁盘, 故障预测, 机器学习

Abstract:

As disk is one of the most important data storage device, it is significant to improve disks’ reliability and data availability. Modern disks adopt the SMART protocol to monitor the internal operating status. We employ machine learning methods, including backpropagation neural networks, decision tree, supported vector machine and nave Bayes to analyze the SMART data of disks, which can predict disk failures. Real SMART data of disks are used in experiments to validate and analyze the effectiveness of those methods, and the effectiveness of different methods is compared. The results show that the decision tree method has best prediction rate while the supported vector machine method has the lowest false alarm rate.

Key words: disk;failure prediction;machine learning

董勇，蒋艳凰，卢宇彤,周恩强. 面向磁盘故障预测的机器学习方法比较[J]. J4, 2015, 37(12): 2200-2207.

DONG Yong，JIANG Yanhuang，LU Yutong，ZHOU Enqiang. Comparison of machine learning methods
for disk failure prediction [J]. J4, 2015, 37(12): 2200-2207.

参考文献

［1］Schroeder B,Gibson G A. Disk failures in the real world:What does an MTTF of 1，000，000 hours mean to you？［C］∥Proc of the 5th USENIX Conference on File and Storage Technologies,2007:286299．
［2］Gobioff H,Ghemawat S T S. The Google file system［C］∥Proc of the 19th ACM Symposium on Operating Systems Principles（SOSP’03）,2003:2943．
［3］Van Renesse R,Schneider F B. Chain replication for supporting high throughout and availability［C］∥Proc of OSDI’04,2004：91104．
［4］Pinheiro E,Weber WD,Andr L,et al. Failure trends in a large disk drive population ［C］∥Proc of the 5th USENIX Conference on File and Storage Technologies,2007:1．
［5］Murray J F,Hughes G F,KreutzDelgado K. Machine learning methods for predicting failures in hard drives:A multipleinstance application ［J］. Journal of Machine Learning Research,2005,6(1):783816．
［6］Jiang Yanhuang,Zhao Qiangli. Machine learning techniques［M］. Beijing:Publishing House of Electronics Industry,2009(in Chinese)
［7］LibEDM［EB/OL］. ［20150313］.https://github.com/QiangliZhao/LibEDM.
［8］http://pan.baidu.com/share/link？shareid=189977&uk=4278294944.
［9］Hughes G F,Murray J F,KreutzDelgado K,et al. Improved disk drive failure warnings［J］. IEEE Transactions on Reliability,2002，51(3):350357．
［10］Wang Yu,Miao Qiang,Ma E W M,et al. Online anomaly detection for hard disk drives based on mahalanobis distance［J］. IEEE Transactions on Reliability,2013,62(1):136145.
［11］Wang Yu,Ma E W M,Tommy W S Chow,et al. A twostep parametric method for failure prediction in hard disk drives［J］. IEEE Transactions on Industrial Informatics,2014,10（1）:419430．
［12］Hamerly G,Elkan C. Bayesian approaches to failure prediction for disk drives［C］∥Proc of the 18th International Conference on Machine Learning （ICML）,2001:202209．
［13］Tan Y,Gu X. On predictability of system anomalies in real world［C］∥Proc of the 18th Annual IEEE/ACM International Symposium on Modeling,Analysis Simulation of Computer and Telecommunication System,2010:1．
［14］Zhao Y,Liu X,Gan S,et al. Predicting disk failure with HMM and HSMMbased approaches［C］∥Proc of the 10th International Conference on Data Mining,2010:390404．
［15］Murray J F, Hughes G F, KreutzDelgado K. Machine learning methods for predicting failures in hard drives:A multiple instance application［J］. Journal of Machine Learning Research,2005,6(1):783816.
［16］Murray J F,Hughes G F,KreutzDelgado K. Hard drive failure prediction using nonparametric statistical methods［C］∥Proc of the ICANN/ICONIP,2003:1．
［17］Zhu Bingpeng,Wang Gang,Liu Xiaoguang，et al. Proactive drive failure prediction for large scale storage systems［C］∥Proc of IEEE Conference on Massive Data Storage Systems and Technologies,2013:15．
［18］Liu Jingning,Rao Guolin,Feng Dan. A S.M.A.R.Tbased method for keeping RAID data’s high dependability［J］. Computer Engineering & Science,2007，29（5）:2123．(in Chinese)
［19］Zhang Chao. Research of selfhealing technique on high performance disk array ［D］.Changsha:Natioanl University of Defense Technology，2008.(in Chinese)
［20］Hu Wei. Research of high reliable disk array technology based on intelligent failure prediction and selfhealing ［D］.Changsha:Natioanl University of Defense Technology，2010．(in Chinese)
附中文参考文献:
［6］蒋艳凰，赵强利. 机器学习方法［M］. 北京:电子工业出版社，2009.
［18］刘景宁，饶国林，冯丹.一种基于S.M.A.R.T的保障RAID数据高可靠性的方法［J］. 计算机工程与科学,2007，29（5）:2123．
［19］张超. 高性能磁盘阵列自修复技术研究［D］.长沙：国防科学技术大学，2008．
［20］胡维. 基于智能预警和自修复的高可靠磁盘阵列关键技术研究［D］.长沙：国防科学技术大学,2010．

[1]	赵振宇, 杨天豪, 蒋汶乘, 张书政. 基于机器学习的多压多温多参标准单元延迟快速计算方法[J]. 计算机工程与科学, 2023, 45(08): 1331-1338.
[2]	李小玲, 方建滨, 马俊, 谭霜, 谭郁松. 基于监督学习的稀疏矩阵自动任务分配[J]. 计算机工程与科学, 2023, 45(05): 782-789.
[3]	胡艳芳, 熊文, 高炜. 基于 Spark 平台的网络游戏用户流失预测方法[J]. 计算机工程与科学, 2022, 44(10): 1730-1737.
[4]	唐阳坤, 鲜港, 杨文祥, 喻杰, 张晓蓉, 王耀彬. 基于用户行为的超级计算机作业失败预测方法[J]. 计算机工程与科学, 2022, 44(10): 1753-1761.
[5]	楚阳, 徐文龙. 基于计算机辅助诊断技术的阿尔兹海默症早期分类研究综述[J]. 计算机工程与科学, 2022, 44(05): 879-893.
[6]	崔弘, 赵双, 张广胜, 苏金树. 基于机器学习的移动代理应用流量识别方法[J]. 计算机工程与科学, 2022, 44(04): 654-664.
[7]	刘国强, 赵振宇, 赵晨煜, 韩奥, 杨天豪. 基于机器学习的PCB布线电阻计算方法[J]. 计算机工程与科学, 2022, 44(03): 396-402.
[8]	李文丽. 基于朴素贝叶斯分类的网络谣言识别研究[J]. 计算机工程与科学, 2022, 44(03): 495-501.
[9]	曹良林, 贲可荣, 张献. 基于代理辅助多目标萤火虫算法的软件缺陷预测方法研究[J]. 计算机工程与科学, 2022, 44(02): 257-265.
[10]	张家灏, 邓科峰, 聂腾飞, 任开军, 宋君强. 基于机器学习的海洋中尺度涡检测识别研究综述[J]. 计算机工程与科学, 2021, 43(12): 2115-2125.
[11]	欧琦媛, 祝恩. 基于压缩子空间对齐的多核聚类算法[J]. 计算机工程与科学, 2021, 43(10): 1730-1735.
[12]	何静, 李晋文, 杨安毅. 基于机器学习方法的高速信道建模研究[J]. 计算机工程与科学, 2021, 43(06): 984-988.
[13]	曾杰, 贲可荣, 张献, 徐永士. 融合文本分布式表示的重复缺陷报告检测[J]. 计算机工程与科学, 2021, 43(04): 670-680.
[14]	张原, 姜焕成. 基于深度学习的大口径火炮健康管理系统研究[J]. 计算机工程与科学, 2020, 42(11): 2050-2058.
[15]	王思齐, 胡婧韬, 余广, 祝恩, 蔡志平. 智能视频异常事件检测方法综述[J]. 计算机工程与科学, 2020, 42(08): 1393-1405.