移动边缘网络中基于双深度Q学习的高能效资源分配方法
Energy-efficient resource allocation method in mobile edge network based on double deep Q-learning
- 通信学报 2020年41卷第12期 页码：148-161
- 作者机构：
1. 北京邮电大学网络与交换技术国家重点实验室，北京 100876
2. 重庆大学微电子与通信工程学院，重庆 400044
- 作者简介：
[ "喻鹏（1986- ），男，湖北随州人，博士，北京邮电大学副教授、博士生导师，主要研究方向为5G/6G网络智能管控等。" ]
[ "张俊也（1998- ），女，山西太原人，北京邮电大学硕士生，主要研究方向为5G/6G网络智能管控、移动边缘计算等。" ]
[ "李文璟（1973- ），女，山西太谷人，博士，北京邮电大学教授、博士生导师，主要研究方向为无线网络管理和自组织网络等。" ]
[ "周凡钦（1988- ），男，河南渑池人，博士，北京邮电大学在站博士后，主要研究方向为无线异构网络的资源管理和负载均衡等。" ]
[ "丰雷（1987- ），男，北京人，北京邮电大学副教授、硕士生导师，主要研究方向为无线网络和智能电网的资源管理等。" ]
[ "付澍（1985- ），男，贵州贵阳人，博士，重庆大学副教授、硕士生导师，主要研究方向为星地通信、NOMA、物联网、网络一体化等。" ]
[ "邱雪松（1973- ），男，江西上饶人，博士，北京邮电大学教授、博士生导师，主要研究方向为网络管理与通信软件等。" ]
- 基金信息：国家重点研发计划基金资助项目(2018YFE0205502)
DOI：10.11959/j.issn.1000-436X.2020218
中图分类号： TN929.5-
网络首发：2020-12，
纸质出版：2020-12-25
- 稿件说明：
移动端阅览
喻鹏, 张俊也, 李文璟, 等. 移动边缘网络中基于双深度Q学习的高能效资源分配方法[J]. 通信学报, 2020,41(12):148-161.
Peng YU, Junye ZHANG, Wenjing LI, et al. Energy-efficient resource allocation method in mobile edge network based on double deep Q-learning[J]. Journal on Communications, 2020, 41(12): 148-161.
喻鹏, 张俊也, 李文璟, 等. 移动边缘网络中基于双深度Q学习的高能效资源分配方法[J]. 通信学报, 2020,41(12):148-161. DOI： 10.11959/j.issn.1000-436X.2020218.
Peng YU, Junye ZHANG, Wenjing LI, et al. Energy-efficient resource allocation method in mobile edge network based on double deep Q-learning[J]. Journal on Communications, 2020, 41(12): 148-161. DOI： 10.11959/j.issn.1000-436X.2020218.
摘要
为了提升移动边缘网络中系统的能量使用效率，面向多任务、多终端设备、多边缘网关、多边缘服务器共存网络架构的下行通信过程，提出了一种基于双深度Q学习（DDQL）的通信、计算、存储融合资源分配方法。以任务平均能耗最小化为优化目标，设置任务时延和通信、计算、存储资源限制等约束条件，构建了对应的资源分配模型。依据模型特征，基于DDQL框架，提出了适用于通信和计算资源智能决策、存储资源按需分配的资源分配模型和算法。仿真结果表明，所提出的基于DDQL资源分配方法可以有效地解决多任务资源分配问题，具有较好的收敛性和较低的时间复杂度，在保障业务服务质量的同时，相对于基于随机算法、贪心算法、粒子群优化算法、深度Q学习等方法，降低了至少5%的任务平均能耗。
Abstract
To improve the system energy efficiency in mobile edge networks
a resource allocation method based on double deep Q-learning(DDQL) for integration of communication
computing
storage resources was proposed for the downlink communication process under the network architecture of multiple tasks
end devices
edge gateways and edge servers.A resource allocation model was constructed
which took the minimization of average energy consumption of tasks as the optimization goal and set the constraints of task delay limits and communication
computing
and storage resource limits.According to the model characteristics
a suitable resource allocation model and method based on DDQL framework was proposed to make intelligent allocation decisions for communication and computing resources and allocate storage resources on demand.Simulation results show that the proposed DDQL-based solution can effectively solve the multi-task resource allocation problem with good converge and low time complexity
and it reduces the average energy consumption of tasks by at least 5% compared with the solving methods based on random algorithm
greedy algorithm
particle swarm optimization algorithm and deep Q-learning while ensuring the quality of service.
关键词
Keywords
references
陈亮 , 余少华 . 6G 移动通信发展趋势初探（特邀） [J ] . 光通信研究 , 2019 , 45 ( 4 ): 1 - 8 .
CHEN L , YU S H . Preliminary Study on the Trend of 6G Mobile Communication (special invitation) [J ] . Study on Optical Communications , 2019 , 45 ( 4 ): 1 - 8 .
田辉 , 范绍帅 , 吕昕晨 , 等 . 面向 5G 需求的移动边缘计算 [J ] . 北京邮电大学学报 , 2017 , 40 ( 2 ): 1 - 10 .
TIAN H , FAN S S , LYU X C , et al . Mobile edge computing for 5G requirements [J ] . Journal of Beijing University of Posts and Telecommunications , 2017 , 40 ( 2 ): 1 - 10 .
STRINATI E C , BARBAROSSA S , GONZALEZ-JIMENEZ J L , et al . 6G:the next frontier:from holographic messaging to artificial intelligence using subterahertz and visible light communication [J ] . IEEE Vehicular Technology Magazine , 2019 , 14 ( 3 ): 42 - 50 .
毕成 . 5G无线网络节能技术研究 [J ] . 电信工程技术与标准化 , 2020 , 33 ( 8 ): 1 - 5 .
BI C . Research on energy saving technology of 5G wireless network [J ] . Telecom Engineering Technics and Standardization , 2020 , 33 ( 8 ): 1 - 5 .
赵亚军 , 郁光辉 , 徐汉青 . 6G移动通信网络:愿景、挑战与关键技术 [J ] . 中国科学:信息科学 , 2019 , 8 ( 49 ): 963 - 987 .
ZHAO Y J , YU G H , XU H Q . 6G mobile communication networks:vision,challenges,and key technologies [J ] . Scientia Sinica (Informationis) , 2019 , 8 ( 49 ): 963 - 987 .
CAI S , CHE Y , DUAN J , et al . Green 5G heterogeneous networks through dynamic small-cell operation [J ] . IEEE Journal on Selected Areas in Communications , 2016 , 34 ( 5 ): 1103 - 1115 .
ZHOU Y , TIAN L , LIU L , et al . Fog computing enabled future mobile communication networks:a convergence of communication and computing [J ] . IEEE Communications Magazine , 2019 , 57 ( 5 ): 20 - 27 .
LIU L , ZHOU Y , YUAN J , et al . Economically optimal MS association for multimedia content delivery in cache-enabled heterogeneous cloud radio access networks [J ] . IEEE Journal on Selected Areas in Communications , 2019 , 37 ( 7 ): 1584 - 1593 .
BI S , HUANG L , ZHANG Y J A . Joint optimization of service caching placement and computation offloading in mobile edge computing systems [J ] . IEEE Transactions on Wireless Communications , 2020 , 19 ( 7 ): 4947 - 4963 .
MAO Y , ZHANG J , LETAIEF K B . Dynamic computation offloading for mobile-edge computing with energy harvesting devices [J ] . IEEE Journal on Selected Areas in Communications , 2016 , 34 ( 12 ): 3590 - 3605 .
LIU L , ZHOU Y , GARCIA V , et al . Load aware joint CoMP clustering and inter-cell resource scheduling in heterogeneous ultra dense cellular networks [J ] . IEEE Transactions on Vehicular Technology , 2017 , 67 ( 3 ): 2741 - 2755 .
BADRI H , BAHREINI T , GROSU D , et al . Risk-based optimization of resource provisioning in mobile edge computing [C ] // 2018 IEEE/ACM Symposium on Edge Computing (SEC) . Piscataway:IEEE Press , 2018 : 328 - 330 .
WANG L , JIAO L , LI J , et al . Moera:mobility-agnostic online resource allocation for edge computing [J ] . IEEE Transactions on Mobile Computing , 2019 , 18 ( 8 ): 1843 - 1856 .
CHEN X , JIAO L , LI W , et al . Efficient multi-user computation offloading for mobile-edge cloud computing [J ] . IEEE/ACM Transactions on Networking , 2015 , 24 ( 5 ): 2795 - 2808 .
ZENG D , GU L , PAN S , et al . Resource management at the network edge:a deep reinforcement learning approach [J ] . IEEE Network , 2019 , 33 ( 3 ): 26 - 33 .
梁应敞 , 谭俊杰 , NIYATO D . 智能无线通信技术研究概况 [J ] . 通信学报 , 2020 , 41 ( 7 ): 1 - 17 .
LIANG Y C , TAN J J , NIYATO D . Overview on intelligent wireless communication technology [J ] . Journal on Communications , 2020 , 41 ( 7 ): 1 - 17 .
XU J , CHEN L , REN S . Online learning for offloading and autoscaling in energy harvesting mobile edge computing [J ] . IEEE Transactions on Cognitive Communications and Networking , 2017 , 3 ( 3 ): 361 - 373 .
ZHANG C , PATRAS P , HADDADI H . Deep learning in mobile and wireless networking:a survey [J ] . IEEE Communications Surveys ＆Tutorials , 2019 , 21 ( 3 ): 2224 - 2287 .
ZHAO N , LIANG Y C , NIYATO D , et al . Deep reinforcement learning for user association and resource allocation in heterogeneous cellular networks [J ] . IEEE Transactions on Wireless Communications , 2019 , 18 ( 11 ): 5141 - 5152 .
LI J , GAO H , LYU T , et al . Deep reinforcement learning based computation offloading and resource allocation for MEC [C ] // 2018 IEEE Wireless Communications and Networking Conference . Piscataway:IEEE Press , 2018 : 1 - 6 .
WANG J , ZHAO L , LIU J , et al . Smart resource allocation for mobile edge computing:a deep reinforcement learning approach [J ] . IEEE Transactions on Emerging Topics in Computing , 2019 ,,doi:10.1109/TETC.2019.2902661.
LI Y , QI F , WANG Z , et al . Distributed edge computing offloading algorithm based on deep reinforcement learning [J ] . IEEE Access , 2020 , 8 : 85204 - 85215 .
VAN H H , GUEZ A , SILVER D . Deep reinforcement learning with double Q-learning [C ] // Proceedings of the AAAI Conference on Artificial Intelligence . Palo Alto:AAAI Press , 2016 : 2094 - 2100 .
CHEN X , ZHANG H , WU C , et al . Optimized computation of-floading performance in virtual edge computing systems via deep reinforcement learning [J ] . IEEE Internet of Things Journal , 2018 , 6 ( 3 ): 4005 - 4018 .
MAO Y , YOU C , ZHANG J , et al . A survey on mobile edge computing:the communication perspective [J ] . IEEE Communications Surveys ＆ Tutorials , 2017 , 19 ( 4 ): 2322 - 2358 .
刘全 , 翟建伟 , 章宗长 , 等 . 深度强化学习综述 [J ] . 计算机学报 , 2018 , 41 ( 1 ): 1 - 27 .
LIU Q , ZHAI J W , ZHANG Z Z , et al . A survey on deep reinforcement learning [J ] . Chinese Journal of Computers , 2018 , 41 ( 1 ): 1 - 27 .
MNIH V , KAVUKCUOGLU K , SILVER D , et al . Human-level control through deep reinforcement learning [J ] . Nature , 2015 , 518 ( 7540 ): 529 - 533 .
CAO X , WANG F , XU J , et al . Joint computation and communication cooperation for energy-efficient mobile edge computing [J ] . IEEE Internet of Things Journal , 2018 , 6 ( 3 ): 4188 - 4200 .
TRAN T X , POMPILI D . Joint task offloading and resource allocation for multi-server mobile-edge computing networks [J ] . IEEE Transactions on Vehicular Technology , 2018 , 68 ( 1 ): 856 - 868 .
YANG X , YU P , FENG L , et al . A deep reinforcement learning based mechanism for cell outage compensation in 5G UDN [C ] // 2019 IFIP/IEEE Symposium on Integrated Network and Service Management . Piscataway:IEEE Press , 2019 : 476 - 481 .
WANG S , PAN C , YIN C . Joint heterogeneous tasks offloading and resource allocation in mobile edge computing systems [C ] // 2018 10th International Conference on Wireless Communications and Signal Processing . Piscataway:IEEE Press , 2018 : 1 - 6 .
ARULKUMARAN K , DEISENROTH M P , BRUNDAGE M , et al . Deep reinforcement learning:a brief survey [J ] . IEEE Signal Processing Magazine , 2017 , 34 ( 6 ): 26 - 38 .
0
浏览量
1228
下载量
0
CSCD
- 网络PDF下载
- XML下载
- Meta-XML下载
- BibTeX下载
- Endnote下载
- RefMan 下载
- NoteExpress下载
- NoteFirst下载
关联资源
相关文章
相关作者
相关机构
AI问答- 技术支持由北京北大方正电子有限公司提供 京ICP备09064830号-19
京公网安备11010802024621 - 本系统建议在Chrome、 IE9+ 以上版本浏览器阅读本站内容，360浏览器请切换至极速模式
- Cookies帮助我们提供服务并提供个性化体验。使用本网站，即表示您同意我们使用Cookies
- 总访问量：584097 今日访问量：1913