• 中国计算机学会会刊
  • 中国科技核心期刊
  • 中文核心期刊

J4 ›› 2013, Vol. 35 ›› Issue (11): 94-99.

• 论文 • 上一篇    下一篇

PEAK:一种面向弱节点集群的并行可演化管理框架

张鲁飞,吴东,谢向辉   

  1. (数学工程与先进计算国家重点实验室,江苏 无锡 214125)
  • 收稿日期:2013-08-06 修回日期:2013-10-18 出版日期:2013-11-25 发布日期:2013-11-25
  • 基金资助:

    国家863计划资助项目(2013AA01A213)

PEAK:Parallel evolutionary administration
framework towards cluster of wimpy nodes  

ZHANG Lu-fei,WU Dong,XIE Xiang-hui   

  1. (State Key Laboratory of Mathematical Engineering and Advanced Computing,Wuxi 214125,China)
  • Received:2013-08-06 Revised:2013-10-18 Online:2013-11-25 Published:2013-11-25

摘要:

弱节点集群Ant II是一种面向低功耗数据密集型计算的体系结构,由若干低功耗嵌入式处理器和固态存储紧耦合而成。面向弱节点集群特殊的应用需求和硬件架构,提出了一种具备自愈、热升级的分布式存储和计算框架PEAK。用原生并行编程语言Erlang开发,利用监控树和代码热替换技术等,保证系统的自愈、可演化;采用了去中心化可伸缩容错的Dynamo架构,保证分布式环境下系统的可用性和最终一致性;提出分布式元服务管理框架,提供高效灵活的基础服务部署与管理,可利用若干元服务快速构建PEAK;提供了key-value的存储方式和基于MapReduce的查询功能。测评结果显示PEAK可以很好地平衡计算和I/O能力,满足大规模并行数据访问需求。

关键词: 弱节点集群, 管理框架, 分布式系统, 演化计算

Abstract:

Ant II is a new cluster architecture for low-power data-intensive computing which consists of big amounts of low-power embedded CPUs and local flash storage. The key contributions of this paper are the principles of the parallel administration framework towards cluster of wimpy nodes and the design and implementation of PEAK, which is an evolutionary, self-healing, hot-plugged, scalable, highly available, and high-performance distributed storage system and computing platform. It is developed with natural parallel programming language Erlang, which supports supervision tree, hot code swapping and sandboxing. It uses decentralized Dynamo architecture which provides great scalability and availability using chain replication on a consistent hashing ring. It builds on distributed meta-service management framework, and therefore can easily evolve. It is not only purely log-structured storage that provides the basis for high performance on flash storage, but also an analytic tool using MapReduce query language. The evaluation demonstrates that PEAK balances computation and I/O capabilities so as to enable efficient, massively parallel access to data.

 

Key words: cluster of wimpy nodes;administration framework;distributed system;evolutionary computation