• 中国计算机学会会刊
  • 中国科技核心期刊
  • 中文核心期刊

J4 ›› 2014, Vol. 36 ›› Issue (11): 2054-2060.

• 论文 • 上一篇    下一篇

基于MapReduce虚拟集群的能耗优化算法

邓聃婷1,滕飞1,2,李天瑞1,杨浩1   

  1. (1.西南交通大学信息科学与技术学院, 四川 成都 610031;
    2.南京大学计算机软件新技术国家重点实验室,江苏 南京 210023)
  • 收稿日期:2014-06-10 修回日期:2014-08-21 出版日期:2014-11-25 发布日期:2014-11-25
  • 基金资助:

    国家自然科学基金资助项目(61202043);网络智能信息处理四川省高校重点实验室开放课题资助项目(SZJJ2014049)

An energy-saving algorithm for
MapReduce-based virtual cluster       

DENG Danting1,TENG Fei1,2,LI Tianrui1,YANG Hao1   

  1. (1.School of Information Science and Technology,Southwest Jiaotong University,Chengdu  610031;
    2.State key Laboratory for Novel Software Technology,Nanjing University,Nanjing 210023,China)
  • Received:2014-06-10 Revised:2014-08-21 Online:2014-11-25 Published:2014-11-25

摘要:

随着全球能源危机的出现,许多研究者开始关注数据中心的能耗问题。在满足用户需求的前提下,减少数据中心的活跃节点个数能够有效地降低其能耗。传统的减少活跃节点的方式是虚拟机迁移,但虚拟机迁移会造成极大的系统开销。提出一种基于MapReduce虚拟集群的能耗优化算法——在线时间平衡算法OTBA,能够减少活跃物理节点数,有效降低数据中心的能耗,并且避免了虚拟机的迁移。通过建立云数据中心的能耗模型、用户提交服务的排队模型和评价作业完成质量的作业运行模型,确定了数据中心节能模型的目标函数和变量因子。在线时间平衡算法是基于虚拟云环境和在线MapReduce作业的一种节能调度算法,能够在虚拟机的生命周期和资源利用率之间做出权衡,使数据中心激活的服务器达到最少,能耗降到最低。此外,该结果通过仿真和Hadoop平台上的实验得到了验证。

关键词: 能耗优化;虚拟机放置;在线;MapReduce;云计算

Abstract:

In the global energy crisis, many researchers begin to pay close attention to the problem of data centers’ energy consumption. On the premise of meeting the users’ demand, reducing the active nodes of adata center can effectively reduce the whole energy consumption. Virtual machine migration is a traditional way to reduce active nodes, but causes huge system cost. An energysaving algorithm for MapReducebased virtual cluster, named Online Time Balance Algorithm (OTBA), is proposed. The proposed algorithm can reduce the number of active physical nodes, reduce the energy consumption effectively, and avoid migrating the virtual machines. The objective function and the variable factors are determined by building the energy consumption model, the queue model and the MapReduce performance model. Since OTBA is based on virtual cloud environment and online MapReduce, it can make a  tradeoff between the runtime of virtual machines and the resource utilization so as to minimize the number of the activated physical servers in the data center and the energy consumption. At last, the algorithm is validated through experiments in simulation environment and on Hadoop platform.

Key words: energy efficiency;VM placement;online;MapReduce;cloud computing