storm平台下工作节点的内存电压调控节能策略

蒲勇霖; 于炯; 鲁亮; 卞琛; 廖彬; 李梓杨

doi:10.11959/j.issn.1000-436x.2018213

您当前的位置：

首页 >

文章列表页 >

storm平台下工作节点的内存电压调控节能策略

学术论文 | 更新时间：2024-06-05

- storm平台下工作节点的内存电压调控节能策略
- Energy-efficient strategy for work node by DRAM voltage regulation in storm
- 通信学报 2018年39卷第10期页码：97-117
- 作者机构：
  
  1. 新疆大学软件学院，新疆乌鲁木齐 830008
  2. 新疆大学信息科学与工程学院，新疆乌鲁木齐 830046
  3. 新疆财经大学统计与信息学院，新疆乌鲁木齐 830012
  4. 新疆润物网络有限公司，新疆乌鲁木齐 830002
- 作者简介：
  
  [ "蒲勇霖（1991−），男，山东淄博人，新疆大学博士生，主要研究方向为内存计算、流式计算、绿色计算等。" ]
  [ "于炯（1964−），男，新疆乌鲁木齐人，博士，新疆大学教授、博士生导师，主要研究方向为并行计算、分布式系统、绿色计算等。" ]
  [ "鲁亮（1990−），男，新疆乌鲁木齐人，新疆大学博士生，主要研究方向为分布式系统、内存计算、绿色计算。" ]
  [ "卞琛（1981−），男，江苏南京人，博士，新疆大学副教授，主要研究方向为分布式系统、内存计算、绿色计算等。" ]
  [ "廖彬（1986−），男，新疆乌鲁木齐人，博士，新疆财经大学副教授、硕士生导师，主要研究方向为分布式系统、数据库理论与技术、绿色计算等。" ]
  [ "李梓杨（1993−），男，新疆乌鲁木齐人，新疆大学硕士生，主要研究方向为流式计算、内存计算等。" ]
- 基金信息：
  
  国家自然科学基金资助项目(61262088);国家自然科学基金资助项目(61462079);国家自然科学基金资助项目(61562086);国家自然科学基金资助项目(61363083);国家自然科学基金资助项目(61562078);国家科技部科技支撑项目(2015BAH02F01);新疆维吾尔自治区研究生科研创新项目(XJGRI2016028)
- DOI：10.11959/j.issn.1000-436x.2018213
  中图分类号： TP311
- 网络出版日期：2018-10，
  
  纸质出版日期：2018-10-25
- 稿件说明：
移动端阅览
蒲勇霖, 于炯, 鲁亮, 等. storm平台下工作节点的内存电压调控节能策略[J]. 通信学报, 2018,39(10):97-117.

Yonglin PU, Jiong YU, Liang LU, et al. Energy-efficient strategy for work node by DRAM voltage regulation in storm[J]. Journal on communications, 2018, 39(10): 97-117.
蒲勇霖, 于炯, 鲁亮, 等. storm平台下工作节点的内存电压调控节能策略[J]. 通信学报, 2018,39(10):97-117. DOI： 10.11959/j.issn.1000-436x.2018213.

Yonglin PU, Jiong YU, Liang LU, et al. Energy-efficient strategy for work node by DRAM voltage regulation in storm[J]. Journal on communications, 2018, 39(10): 97-117. DOI： 10.11959/j.issn.1000-436x.2018213.

摘要

针对传统大数据流式计算平台节能策略并未考虑数据处理及传输的实时性问题，首先根据数据流处理的特点与storm集群的结构，建立有向无环图、实例并行度、任务资源分配与关键路径模型。其次结合拓扑执行关键路径与系统性能的分析，提出一种 storm 平台下工作节点的内存电压调控节能策略(WNDVR-storm

energy-efficient strategy for work node by dram voltage regulation in storm)，该策略针对是否有工作节点位于拓扑执行的非关键路径上设计了 2 种节能算法。最后根据系统数据处理及传输的制约条件确定工作节点 CPU 使用率与数据传输量的阈值，并对选定的工作节点内存电压做出动态调整。实验结果表明，该策略能有效降低能耗，且制约条件越小节能效率越高。

Abstract

Focused on the problem that traditional energy-efficient strategies never consider about the real time of data processing and transmission

models of directed acyclic graph

parallelism of instance

resource allocation for task and critical path were set up based on the features of data stream processing and the structure of storm cluster.Meanwhile

the WNDVR-storm (energy-efficient strategy for work node by dram voltage regulation in storm) was proposed according to the analysis of critical path and system performance

which included two energy-efficient algorithms aiming at whether there were any work nodes executing on the non-critical path of a topology.Finally

the appropriate threshold values fit for the CPU utilization of work node and the volume of transmitted data were determined based on the data processing and transmission constraints to dynamically regulate the DRAM voltage of the system.The experimental result shows that the strategy can reduce energy consumption effectively.Moreover

the fewer constraints are

the higher energy efficiency is.

关键词

Keywords

references

孟小峰 , 慈祥 . 大数据管理:概念、技术与挑战 [J ] . 计算机研究与发展 , 2013 , 50 ( 1 ): 146 - 169 .

MENG X F , CI X . Big data management:concepts,techniques and challenges [J ] . Journal of Computer Research and Development , 2013 , 50 ( 1 ): 146 - 169 .

孙大为 . 大数据流式计算:应用特征和技术挑战 [J ] . 大数据 , 2015 , 1 ( 3 ): 99 - 105 .

SUN D W . Big data stream comuting:features and challenges [J ] . Big Data Research , 2015 , 1 ( 3 ): 99 - 105 .

CHEN C L P , ZHANG C Y . Data-intensive applications,challenges,techniques and technologies:a survey on big data [J ] . Information Sciences , 2014 , 275 ( 11 ): 314 - 347 .

RANJAN R . Streaming big data processing in datacenter clouds [J ] . IEEE Cloud Computing , 2014 , 1 ( 1 ): 78 - 83 .

KAMBATLA K , KOLLIAS G , KUMAR V , et al . Trends in big data analytics [J ] . Journal of Parallel and Distributed Computing , 2014 , 74 ( 7 ): 2561 - 2573 .

邓维 , 刘方明 , 金海 , 等 . 云计算数据中心的新能源应用:研究现状与趋势 [J ] . 计算机学报 , 2013 , 36 ( 3 ): 582 - 598 .

DENG W , LIU F M , JIN H , et al . Leveraging renewable energy in cloud computing datacenters:state of the art and future research [J ] . Chinese Journal of Computers , 2013 , 36 ( 3 ): 582 - 598 .

廖彬 , 张陶 , 于炯 , 等 . 温度感知的 MapReduce 节能任务调度策略 [J ] . 通信学报 , 2016 , 37 ( 1 ): 61 - 75 .

LIAO B , ZHANG T , YU J , et al . Temperature aware energy-efficient task scheduling strategies for MapReduce [J ] . Journal on Communications , 2016 , 37 ( 1 ): 61 - 75 .

蒲勇霖 , 于炯 , 鲁亮 , 等 . 基于实时流式计算系统的数据分类节能策略 [J ] . 计算机工程与设计 , 2017 , 38 ( 1 ): 59 - 64 .

PU Y L , YU J , LU L , et al . Energy-efficient strategy based on data classification in real-time stream computing system [J ] . Computer Engineering and Design , 2017 , 38 ( 1 ): 59 - 64

SUN D , ZHANG G , YANG S , et al . Re-Stream:Real-time and energy-efficient resource scheduling in big data stream computing environments [J ] . Information Sciences , 2015 ,( 319 ): 92 - 112 .

TOSHNIWAL A , TANEJA S , SHUKLA A , et al . Storm @Twitter [C ] // Proc of the 2014 ACM SIGMOD Int Conf on Management of data . ACM , 2014 : 147 - 156 .

BORTHAKUR D , GRAY J , SARMA J S , et al . Apache hadoop goes realtime at Facebook [C ] // The 2011 ACM SIGMOD Int Conf on Management of data . 2011 : 1071 - 1080 .

NEUMEVER L , ROBBINS B , NAIR A , et al . S4:Distributed stream computing platform [C ] // The 10th IEEE Int Conf on Data Mining Workshops (ICDMW 2010) . 2010 : 170 - 177 .

ZAHARIA M , DAS T , LI H , et al . Discretized streams:an efficient and fault-tolerant model for stream processing on large clusters [C ] // The 4th USENIX conf on Hot Topics in Cloud Computing . 2012 :10.

FISCHER M J , SU X , YIN Y . Assigning tasks for efficiency in hadoop [C ] // The 22th annual ACM Symp on Parallelism in algorithms and architectures . 2010 : 30 - 39 .

ALBERS S . Energy-efficient algorithms [J ] . Communications of the ACM , 2010 , 53 ( 5 ): 86 - 96 .

TCHEMYKH A , PECERO J E , BARRONDO A , et al . Adaptive energy efficient scheduling in Peer-to-Peer desktop grids [J ] . Future Generation Computer Systems , 2014 , 36 ( 7 ): 209 - 220 .

POLLAKIS E , CAVALCANTE R L G , STANCZAK S . Traffic demand-aware topology control for enhanced energy-efficiency of cellular networks [J ] . EURASIP Journal on Wireless Communications and Networking , 2016 , 2016 ( 1 ): 61 - 77 .

LIU G X , XU J L , HONG X B . Internet of things sensor node information scheduling model and energy saving strategy [J ] . Advanced Materials Research , 2013 , 773 ( 1 ): 215 - 220 .

WANG X , WANG Y , CUI Y . A new multi-objective bi-level programming model for energy and locality aware multi-job scheduling in cloud computing [J ] . Future Generation Computer Systems , 2014 , 36 ( 7 ): 91 - 101 .

HE F Y , WANG F . Research on energy saving for multi-back after blending water from one station system’s parameters optimization [J ] . Advanced Materials Research , 2014 , 1023 : 187 - 191 .

MILI M R , MUSAVIAN L , HAMDI K A , et al . How to increase energy efficiency in cognitive radio networks [J ] . IEEE Transactions on Communications , 2016 , 64 ( 5 ): 1829 - 1843 .

CHEN X , YANG H , ZHANG W . A comprehensive sensitivity study of major passive design parameters for the public rental housing development in Hong Kong [J ] . Energy , 2015 ,( 93 ): 1804 - 1818 .

WANG H , CHEN Q . A semi-empirical model for studying the impact of thermal mass and cost-return analysis on mixed-mode ventilation in office buildings [J ] . Energy ＆ Buildings , 2013 , 67 ( 4 ): 267 - 274 .

YANG T , MINO G , BAROLLI L , et al . Energy-saving in wireless sensor networks considering mobile sensor nodes [J ] . Computer Systems Science ＆ Engineering , 2011 , 27 ( 5 ): 317 - 326 .

孙大为 , 张广艳 , 郑纬民 . 大数据流式计算:关键技术及系统实例 [J ] . 软件学报 , 2014 , 25 ( 4 ): 839 - 862 .

SUN D W , ZHANG G Y , ZHENG W M . Big data stream computing:Technologies and instances [J ] . Journal of Software , 2014 , 25 ( 4 ): 839 - 862 .

LI K C , JIANG H , YANG L T , et al . Big data:algorithms,analytics,and applications [M ] . Florida : CRC PressPress , 2015 : 193 - 214 .

BONAMY R , BILAVARN S , MULLER F . An energy-aware scheduler for dynamically reconfigurable multi-core systems [C ] // International Symposium on Reconfigurable Communication-Centric Systems-On-Chip . 2015 : 1 - 6 .

蒲勇霖 , 于炯 , 鲁亮 , 等 . 大数据流式计算环境下的内存节能策略 [J ] . 小型微型计算机系统 , 2017 , 38 ( 9 ): 1988 - 1993 .

PU Y L , YU J , LU L , et al . Energy-efficient strategy for memory in big data stream computing environment [J ] . Journal of Chinese Mini-Micro Computer Systems , 2017 , 38 ( 9 ): 1988 - 1993 .

TRIHINAS D , PALLIS G , DIKAIAKOS M D . JCatascopia:monitoring elastically adaptive applications in the cloud [C ] // The 14th IEEE/ACM Int Symp on Cluster,Cloud and Grid Computing (CCGrid) . 2014 : 226 - 235 .

VAN D V J S , VAN D W B , LAZOVIK E , et al . dynamically scaling apache storm for the analysis of streaming data [C ] // The 2015 IEEE 1st Int Conf on Big Data Computing Service and Applications . 2015 : 154 - 161 .

CORDESCHI N , SHOJAFAR M , AMENDOLA D , et al . Energy-efficient adaptive networked datacenters for the QoS support of real-time applications [J ] . The Journal of Supercomputing , 2014 , 71 ( 2 ): 448 - 478 .

BASKIYAR S , ABDEL-KADER R . Energy aware DAG scheduling on heterogeneous systems [J ] . Cluster Computing , 2010 , 13 ( 4 ): 373 - 383 .

HSU C H , SLAGTER K D , CHEN S C , et al . Optimizing energy consumption with task consolidation in clouds [J ] . Information Sciences , 2014 , 258 ( 3 ): 452 - 462 .

KIM N , CHO J , SEO E . Energy-credit scheduler:an energy-aware virtual machine scheduler for cloud systems [J ] . Future Generation Computer Systems , 2014 , 32 ( 2 ): 128 - 137 .

ZONG Z , MANZANARES A , RUAN X , et al . EAD and PEBD:two energy-aware duplication scheduling algorithms for parallel tasks on homogeneous clusters [J ] . IEEE Transactions on Computers , 2010 , 60 ( 3 ): 360 - 374 .

PATAN R , RAJASEKHARA B M . Re-storm:real-time energy efficient data analysis adapting storm platform [J ] . Jurnal Teknologi , 2016 , 78 ( 10 ): 139 - 146 .

鲁亮 , 于炯 , 卞琛 , 等 . 大数据流式计算框架Storm的任务迁移策略 [J ] . 计算机研究与发展 , 2018 , 55 ( 1 ): 71 - 92 .

LU L , YU J , BIAN C , et al . A Task Migration Strategy in Big DataStream Computing with Storm [J ] . Journal of Computer Research and Development , 2018 , 55 ( 1 ): 71 - 92 .

MATTEIS T D , MENCAGLI G . Keep calm and react with foresight:strategies for low-latency and energy-efficient elastic data stream processing [J ] . Journal of Systems and Software , 2016 , 51 ( 8 ): 1 - 12 .

PAMLEY M R , ORRO J M , KEOWN W F , et al . Dynamic memory voltage scaling for power management:US,US20100250981 [P ] . 2010 .

蒲勇霖 , 于炯 , 王跃飞 , 等 . 大数据流式计算环境下的阈值调控节能策略 [J ] . 计算机应用 , 2017 , 37 ( 6 ): 1580 - 1586 .

PU Y L , YU J , WANG Y F , et al . Energy-efficient strategy for threshold control in big data stream computing environment [J ] . Journal of Computer Applications , 2017 , 37 ( 6 ): 1580 - 1586 .

HONG S P , YOO S J . Dynamic voltage scaling method of CPU using workload estimator and computer readable medium storing the method:US,US7685446 [P ] . 2010 .

HWANG I , PEDRAM M . A comparative study of the effectiveness of CPU consolidation versus dynamic voltage and frequency scaling in a virtualized multicore server [J ] . IEEE Transactions on Very Large Scale Integration Systems , 2016 , 24 ( 6 ): 2103 - 2116 .

林闯 , 田源 , 姚敏 . 绿色网络和绿色评价:节能机制、模型和评价 [J ] . 计算机学报 , 2011 , 34 ( 4 ): 593 - 612 .

LIN C , TIAN Y , YAO M . Green network and green evaluation:mechanism,modeling and evaluation [J ] . Chinese Journal of Computers , 2011 , 34 ( 4 ): 593 - 612 .

GRIFFITHS N . Nmon performance:A free tool to analyze AIX and Linux performance [EB/OL ] .(2017-06-25)[2017-07-03 ] . http://www.ibm.com/developerworks/aix/library/au-analyze_aix http://www.ibm.com/developerworks/aix/library/au-analyze_aix .

浏览量

1065

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于Storm平台的数据迁移合并节能策略

信息年龄和能耗联合优化的无线体域网卸载和调度策略

流式大数据平台下的弹性数据迁移能效优化策略

多接入边缘计算中相关性任务的联合调度算法

基于信息熵与遗传算法的并行关联规则增量挖掘算法