Design and implementation of multi-tenant oriented lightweight virtual HPC cluster based on supercomputing environment

Yu-song TAN; Rong-zhen LI; Qing-bo WU; Jian-feng ZHANG; Yao-xue ZHANG

doi:10.11959/j.issn.1000-436x.2017279

您当前的位置：

首页 >

文章列表页 >

Design and implementation of multi-tenant oriented lightweight virtual HPC cluster based on supercomputing environment

Papers | 更新时间：2024-06-05

- Design and implementation of multi-tenant oriented lightweight virtual HPC cluster based on supercomputing environment
- Journal on Communications Vol. 38, Issue Z2, Pages: 56-66(2017)
- 作者机构：
  
  1. 国防科技大学计算机学院，湖南长沙 410073
  2. 中南大学信息科学与工程学院，湖南长沙 410083
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.1000-436x.2017279
  CLC： TP338
- Online First：2017-11，
  
  Published：25 November 2017
- 稿件说明：
移动端阅览
Yu-song TAN, Rong-zhen LI, Qing-bo WU, et al. Design and implementation of multi-tenant oriented lightweight virtual HPC cluster based on supercomputing environment[J]. Journal on Communications, 2017, 38(Z2): 56-66.
DOI：

Yu-song TAN, Rong-zhen LI, Qing-bo WU, et al. Design and implementation of multi-tenant oriented lightweight virtual HPC cluster based on supercomputing environment[J]. Journal on Communications, 2017, 38(Z2): 56-66. DOI： 10.11959/j.issn.1000-436x.2017279.

摘要

为了给用户提供按需使用的 HPC 服务并解决用户应用部署的软件依赖性问题，在不破坏现有超大规模高性能集群管理的前提下，基于Fat-Tree网络拓扑和虚拟集群模型，通过对部分资源进行云化管理，设计并实现了一种轻量级的HPC集群交付模式。从而为面向租户需求的高性能应用提供一种云化的HPC租用服务，并解决了HPC应用软件栈的僵化问题，以易用的方式为更多的HPC租户提供服务。实验结果表明，该方法以近似物理节点的性能使HPC具备了云的按需使用的特性，并具有较为理想的植入效率。

Abstract

In order to deliver on-demand HPC services for multi-tenants and solve software dependency problems which rigidly restrict the usage of HPC applications.On the premise of not destroying the existing large scale high performance cluster management

a lightweight HPC cluster delivery model was designed and implement with the Fat-Tree network topology and the virtual HPC cluster model.So as to supply a cloud based HPC rental service for the high performance applications of the multi-tenants and solve the rigid dependable problem of HPC application software stack for more HPC tenants in an easy way

the experimental results show that the proposed method is similar to the performance of the physical nodes

so that the HPC has the on-demand characteristics of the cloud and has a better embedding efficiency.

关键词

Keywords

references

ABDELBAKY M , PARASHAR M , JORDAN K E , et al . Enabling high-performance computing as a service [J ] . IEEE Computer , 2012 , 45 ( 10 ): 72 - 80 .

NIU S , ZHAI J , MA X , et al . Cost-effective cloud HPC resource provisioning by building semi-elastic virtual clusters [C ] // Proceedings of SC13,Networking,Storage and Analysis . 2013 .

XAVIER M , NEVES M , ROSSI , et al . Performance evaluation of container-based virtualization for high performance computing environments [C ] // PDP,21st Euromicro International Conference on , 2013 : 233 - 240 .

SLOMINSKI A , MUTHUSAMY V , KHALAF R . Building a multi-tenant cloud service from legacy code with docker containers [C ] // Proc IEEE IC2E , 2015 : 394 - 396 .

MERKEL D . Docker:lightweight linux containers for consistent development and deployment [J ] . Linux Journal , 2014 , 2014 ( 239 ):2.

YOO A B , JETTE M A , GRONDONA M . Slurm:simple Linux utility for resource management [C ] // Workshop on Job Scheduling Strategies for Parallel Processing . Springer,Berlin,Heidelberg , 2003 : 44 - 60 .

PANG Z , WANG K , XIE M , et al . The TH express-2 high performance interconnect networks [J ] . Frontiers of Computer Science , 2014 , 8 ( 3 ): 357 - 366 .

SEFRAOUI O , AISSAOUI M , ELEULDJ M . Openstack:toward an open-source solution for cloud computing [J ] . International Journal of Computer Applications , 2012 , 55 ( 3 ).

YU M , YI Y , REXFORD J , et al . Rethinking virtual network embedding:substrate support for path splitting and migration [J ] . ACM SIGCOMM Comput.Commun.Rev. , 2008 , 38 ( 2 ): 17 - 29 .

TANTAWI A N , . A scalable algorithm for placement of virtual clusters in large data centers [C ] // Proc.Int’l Symp . IEEE 20th MASCOTS , 2012 : 3 - 10 .

WEI X , LI H . Topology-aware partial virtual cluster mapping algorithm on shared distributed infrastructures [J ] . IEEE Transactions on Parallel and Distributed Systems , 2013 : 2721 - 2730 .

LI X , WANG H , DING B , et al . Resource allocation with multi-factor node ranking in data center networks [J ] . Future Generation Computer Systems , 2014 , 32 : 1 - 12 .

GONG S Q , CHEN J , KANG Q Y , et al . An efficient and coordinated mapping algorithm in virtualized SDN networks [J ] . Front Inform Technol Electron Eng , 2016 , 17 ( 7 ): 701 - 716 .

PAPAGIANNI C , ANDROULIDAKIS G , PAPAVASSILIOU S . Virtual topology mapping in SDN-enabled clouds [C ] // NCCA . 2014

WANG Z , HAN Y , LIN T , et al . Virtual network embedding by exploiting topological information [C ] // IEEE GLOBECOM , 2012 .

LIAO J , FENG M , LI T , et al . Topology-aware virtual network embedding using multiple characteristics [J ] . KSII Trans Internet Inf Syst (TIIS) , 2014 , 8 ( 1 ): 145 - 164 .

CHENG X , SU S , ZHANG Z , et al . Virtual network embedding through topology-aware node ranking [C ] // SIGCOMM Comput Commun . 2011 : 38 - 47 .

LI R Z , WU Q B , TAN Y S , et al . A cost-effective approach of building multi-tenant oriented lightweight virtual HPC cluster [C ] // Cloud Computing and Big Data,CCBD , 2016 .

孙大为 , 常桂然 , 李凤云 , 等 . 一种基于免疫克隆的偏好多维 QoS云资源调度优化算法 [J ] . 电子学报 , 2011 , 39 ( 8 ): 1824 - 1831 .

SUN D W , CHANG G R , LI F Y , et al . Optimizing multi-dimensional QoS cloud resource scheduling by immune clonal with preference [J ] . Acta Electronica Sinica , 2011 , 39 ( 8 ): 1824 - 1831 .

张尧学 , 周悦芝 . 一种云计算操作系统TransOS:基于透明计算的设计与实现 [J ] . 电子学报 , 2011 , 39 ( 5 ): 985 - 990 .

ZHANG Y X , ZHANG Y X , ZHOU Y Z . A new cloud operating system:design and implementation based on transparent computing [J ] . Acta Electronica Sinica , 2011 , 39 ( 8 ): 1824 - 1831 .

程祥 , 张忠宝 , 苏森 , 等 . 虚拟网络映射问题研究综述 [J ] . 通信学报 , 2011 , 32 ( 10 ): 143 - 151 .

CHENG X , ZHANG Z B , SU S , et al . Survey of virtual network embedding problem [J ] . Journal on Communications , 2011 , 32 ( 10 ): 143 - 151 .

江逸茗 , 兰巨龙 , 程东年 , 等 . 分布式环境中基于协商的虚拟网映射算法 [J ] . 通信学报 , 2014 , 35 ( 12 ): 62 - 69 .

JIAN Y M , LAN J L , CHENG D N , et al . Virtual network embedding algorithm based on negotiation in distributed enlivenment [J ] . Journal on Communications , 2014 , 35 ( 12 ): 62 - 69 .

Views

2206

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Research on DDoS detection in multi-tenant cloud based on entropy change

Research on resource monitoring and billing mechanisms of application engine in cloud computing environment

Related Author

Miao WANG

Li-ming WANG

Zhen XU

Duo-he MA

Yi REN

Jing ZHANG

Hong CHEN

Qing-bo WU

Related Institution

College of Computer Science,National University of Defense Technology

Tencent Research Institute

AI问答

⁰