满足本地化计算的集群资源调度策略

郭平; 宁立江; 陈海珠

doi:10.3969/j.issn.1000-436x.2014.z2.001

您当前的位置：

首页 >

文章列表页 >

满足本地化计算的集群资源调度策略

学术论文 | 更新时间：2024-06-05

- 满足本地化计算的集群资源调度策略
- Scheduling strategy for achieving locality in cluster
- 通信学报 2014年35卷第Z2期页码：1-8
- 作者机构：
  
  1. 重庆大学计算机学院，重庆 400044
  2. 软件理论与技术重庆市重点实验室，重庆 400044
  3. 重庆电子工程职业学院，重庆 401331
- 作者简介：
  
  [ "郭平（1963-），男，四川眉山人，博士，重庆大学教授，主要研究方向为膜计算、云计算、智能信息系统。" ]
  [ "宁立江（1988-），男，山东莘县人，重庆大学硕士生，主要研究方向为云计算、智能信息系统。" ]
  [ "陈海珠（1980-），女，广西柳州人，博士，重庆电子工程职业学院副教授，主要研究方向为云计算、膜计算。" ]
- 基金信息：
  
  国家自然科学青年基金资助项目(61201347);重庆市自然基金资助项目(2012jjA40022);重庆市自然基金资助项目(2011jjA40027);重庆市自然基金资助项目(2012jjA40011)
- DOI：10.3969/j.issn.1000-436x.2014.z2.001
  中图分类号： TP301.6
- 网络出版日期：2014-11，
  
  纸质出版日期：2014-11-30
- 稿件说明：
移动端阅览
郭平, 宁立江, 陈海珠. 满足本地化计算的集群资源调度策略[J]. 通信学报, 2014,35(Z2):1-8.

Ping GUO, Li-jiang NING, Hai-zhu CHEN. Scheduling strategy for achieving locality in cluster[J]. Journal on communications, 2014, 35(Z2): 1-8.
郭平, 宁立江, 陈海珠. 满足本地化计算的集群资源调度策略[J]. 通信学报, 2014,35(Z2):1-8. DOI： 10.3969/j.issn.1000-436x.2014.z2.001.

Ping GUO, Li-jiang NING, Hai-zhu CHEN. Scheduling strategy for achieving locality in cluster[J]. Journal on communications, 2014, 35(Z2): 1-8. DOI： 10.3969/j.issn.1000-436x.2014.z2.001.

摘要

将本地化计算分成节点本地化计算和机架本地化计算2个层次，分别称作节点本地化计算和机架本地化计算。结合主导资源公平调度策略 DRF 和 Delay 调度约束机制提出一种满足本地化计算的集群资源调度策略DDRF，旨在既能达到一个较高的本地化计算水平，又能保证资源调度的“公平性”。通过数据实验分析了在DDRF调度策略下本地化计算延迟对作业执行效率的影响。

Abstract

The data locality is divided into two levels.One is called the node data locality

which placing tasks on nodes that contain their input data.The other one is called the rack data locality

which placing tasks on nodes whose rack contains their input data.A new scheduling strategy called DDRF is proposed which combines the DRF and the delay.The DDRF is not only able to meet high locality but also achieve fairness.In the DDRF

the simulation results show the influence on the efficiency of jobs’ implement.

关键词

Keywords

references

FIFO [EB/OL ] . http://en.wikipedia.org/wiki/FIFO. http://en.wikipedia.org/wiki/FIFO. .

LEBOUDEC J Y . Rate adaptation,congestion control and fairnes s:a tutorial [EB/OL ] . http://moodle.epfl.ch/file.php/523/CC_Tutorial/cc.pdf http://moodle.epfl.ch/file.php/523/CC_Tutorial/cc.pdf .

Capacity scheduler [EB/OL ] . http://hadoop.apache.org/docs/r0.19.1/c apacity_scheduler.html http://hadoop.apache.org/docs/r0.19.1/c apacity_scheduler.html .

ZAHARIA M , BORTHAKUR D , SARMA J S , et al . Job Scheduling for Multi-user Mapreduce Clusters:UCB/EECS-2009-55 [R ] . Berkeley:EECS Department,University of California , 2009 .

GHODSI A , ZAHARIA M , HINDMAN B , et al . Dominant resource fairness:fair allocation of multiple resource types [A ] . Proceedings of the 8th USENIX Conference on NSDI [C ] . Berbeley:USENIX Association , 2011 . 24 - 37 .

ZAHARIA M , KONWINSKI A , JOSEPH A D , et al . Improving mapreduce performance in heterogeneous environments [A ] . 8th USENIX Symposium on Operating Systems Design and Implementation [C ] . San Diego:USENIX Association , 2008 . 29 - 42 .

POLO J , CARRERA D , BECERRA Y , et al . Performance-driven task co-scheduling for mapreduce environments [A ] . Network Operations and Management Symposium (NOMS) [C ] . San Diego:IEEE , 2010 . 373 - 380 .

KC K , ANYANWU K . Scheduling hadoop jobs to meet deadlines [A ] . Cloud Computing Technology and Science (CloudCom) [C ] . San Diego:IEEE , 2010 : 388 - 392 .

Dynamic priority scheduling [EB/OL ] . http://en.wikipedia.org/wiki/Category:Scheduling_algorithms http://en.wikipedia.org/wiki/Category:Scheduling_algorithms .

ZAHARIA M , BORTHAKUR D , SEN SARMA J , et al . Delay scheduling:a simple technique for achieving locality and fairness in cluster scheduling [A ] . Proceedings of the 5th European conference on Computer systems [. Paris:ACM , 2010 : 265 - 278 .

SHVACHKO K , KUANG H , RADIA S , et al . The hadoop distributed file system [A ] . Mass Storage Systems and Technologies (MSST) [C ] . Lake Tahoe,Nevada,USA:IEEE , 2010 : 1 - 10 .

GUO P , NING L , SU L , et al . A new strategy of resource management for cloud computing [J ] . Information Technology Journal , 2013 , 12 ( 18 ).

Amazon elastic compute cloud [EB/OL ] . http://aws.amazon.com/cn/ec2/ http://aws.amazon.com/cn/ec2/ .

OSTERMANN S , IOSUP A , YIGITBASI N , et al . A Performance Analysis of EC2 Cloud Computing Services for Scientific Computing Cloud Computing [M ] . Springer Berlin Heidelberg , 2010 . 115 - 131 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

暂无数据