基于MapReduce虚拟集群的能耗优化算法

J4 ›› 2014, Vol. 36 ›› Issue (11): 2054-2060.

基于MapReduce虚拟集群的能耗优化算法

邓聃婷1，滕飞1,2，李天瑞1，杨浩1

(1.西南交通大学信息科学与技术学院，四川成都 610031;
2.南京大学计算机软件新技术国家重点实验室,江苏南京 210023)

收稿日期:2014-06-10 修回日期:2014-08-21 出版日期:2014-11-25 发布日期:2014-11-25
基金资助:
国家自然科学基金资助项目(61202043);网络智能信息处理四川省高校重点实验室开放课题资助项目（SZJJ2014049）

An energy-saving algorithm for
MapReduce-based virtual cluster

DENG Danting1,TENG Fei1,2,LI Tianrui1,YANG Hao1

(1.School of Information Science and Technology,Southwest Jiaotong University,Chengdu 610031;
2.State key Laboratory for Novel Software Technology,Nanjing University,Nanjing 210023,China)

Received:2014-06-10 Revised:2014-08-21 Online:2014-11-25 Published:2014-11-25

摘要/Abstract

摘要：

随着全球能源危机的出现，许多研究者开始关注数据中心的能耗问题。在满足用户需求的前提下，减少数据中心的活跃节点个数能够有效地降低其能耗。传统的减少活跃节点的方式是虚拟机迁移，但虚拟机迁移会造成极大的系统开销。提出一种基于MapReduce虚拟集群的能耗优化算法——在线时间平衡算法OTBA，能够减少活跃物理节点数，有效降低数据中心的能耗，并且避免了虚拟机的迁移。通过建立云数据中心的能耗模型、用户提交服务的排队模型和评价作业完成质量的作业运行模型，确定了数据中心节能模型的目标函数和变量因子。在线时间平衡算法是基于虚拟云环境和在线MapReduce作业的一种节能调度算法，能够在虚拟机的生命周期和资源利用率之间做出权衡，使数据中心激活的服务器达到最少，能耗降到最低。此外，该结果通过仿真和Hadoop平台上的实验得到了验证。

关键词: 能耗优化；虚拟机放置；在线；MapReduce；云计算

Abstract:

In the global energy crisis, many researchers begin to pay close attention to the problem of data centers’ energy consumption. On the premise of meeting the users’ demand, reducing the active nodes of adata center can effectively reduce the whole energy consumption. Virtual machine migration is a traditional way to reduce active nodes, but causes huge system cost. An energysaving algorithm for MapReducebased virtual cluster, named Online Time Balance Algorithm (OTBA), is proposed. The proposed algorithm can reduce the number of active physical nodes, reduce the energy consumption effectively, and avoid migrating the virtual machines. The objective function and the variable factors are determined by building the energy consumption model, the queue model and the MapReduce performance model. Since OTBA is based on virtual cloud environment and online MapReduce, it can make a tradeoff between the runtime of virtual machines and the resource utilization so as to minimize the number of the activated physical servers in the data center and the energy consumption. At last, the algorithm is validated through experiments in simulation environment and on Hadoop platform.

Key words: energy efficiency;VM placement;online;MapReduce;cloud computing

邓聃婷1，滕飞1,2，李天瑞1，杨浩1. 基于MapReduce虚拟集群的能耗优化算法[J]. J4, 2014, 36(11): 2054-2060.

DENG Danting1,TENG Fei1,2,LI Tianrui1,YANG Hao1. An energy-saving algorithm for
MapReduce-based virtual cluster [J]. J4, 2014, 36(11): 2054-2060.

参考文献［8］

［1］	Beloglazov A, Abawajy J, Buyya R. Energyaware resource allocation heuristics for efficient management of data centers for cloud computing［J］. Future Generation Computer Systems, 2012, 28(5):755768.
［2］	Chase J S,Anderson D C,Thakar P N, et al. Managing energy and server resources in hosting centers［J］. ACM SIGOPS Operating Systems Review,2001,35(5):103116.
［3］	Dong Jiankang,Wang Hongbo.Issa environment to improve energy efficiency and network performance method for virtual machine placement［J］.Communications Journal, 2014,34(1):7881.(in Chinese)
［4］	Liu H,Jin H,Xu CZ, et al. Performance and energy modeling for live migration of virtual machines［J］. Cluster computing, 2013, 16(2):249264.
［5］	Wirtz T, Ge R. Improving MapReduce energy efficiency for computation intensive workloads［C］∥Proc of the IEEE International Green Computing Conference and Workshops (IGCC),2011:18.
［6］	Xie R,Jia X,Yang K,et al. Energy saving virtual machine allocation in cloud computing［C］∥Proc of the IEEE 33rd International Conference on Distributed Computing Systems Workshops (ICDCSW), 2013:132137.
［7］	Leverich J, Kozyrakis C. On the energy inefficiency of Hadoop clusters［J］. ACM SIGOPS Operating Systems Review, 2010, 44(1):6165.
［8］	Li Qiang, Hao Qinfen, Xiao Limin, et al. Adaptive management and multiobjective optimization for virtual machine placement in cloud computing［J］. Chinese Journal of Computer, 2011,34(12):22532264.(in Chinese)
［9］	Teng Fei, Deng Danting, Yu Lei, et al. An energyefficient VM placement in cloud datacenter［C］∥Proc of 2014 IEEE International Conference on High Performance Computing and Communications (HPCC), 2014:173180.
［10］	Cardosa M, Singh A,Pucha H,et al.Exploiting spatiotemporal tradeoffs for energyaware MapReduce in the cloud［J］.IEEE Transactions on Computers,2012,61(12):17371751.
	附中文参考文献:
［3］	董建康, 王洪波. Issa环境下改进能源效率和网络性能的虚拟机放置方法［J］. 通信学报, 2014, 34(1):7181.
［8］	李强, 郝沁汾, 肖利民. 云计算中虚拟机放置的自适应管理与多目标优化［J］. 计算机学报, 2011, 34(12):22532264.