移动边缘网络中基于双深度Q学习的高能效资源分配方法

喻鹏; 张俊也; 李文璟; 周凡钦; 丰雷; 付澍; 邱雪松

doi:10.11959/j.issn.1000-436X.2020218

您当前的位置：

首页 >

文章列表页 >

移动边缘网络中基于双深度Q学习的高能效资源分配方法

学术论文 | 更新时间：2024-06-05

- 移动边缘网络中基于双深度Q学习的高能效资源分配方法
- Energy-efficient resource allocation method in mobile edge network based on double deep Q-learning
- 通信学报 2020年41卷第12期页码：148-161
- 作者机构：
  
  1. 北京邮电大学网络与交换技术国家重点实验室，北京 100876
  2. 重庆大学微电子与通信工程学院，重庆 400044
- 作者简介：
  
  [ "喻鹏（1986- ），男，湖北随州人，博士，北京邮电大学副教授、博士生导师，主要研究方向为5G/6G网络智能管控等。" ]
  [ "张俊也（1998- ），女，山西太原人，北京邮电大学硕士生，主要研究方向为5G/6G网络智能管控、移动边缘计算等。" ]
  [ "李文璟（1973- ），女，山西太谷人，博士，北京邮电大学教授、博士生导师，主要研究方向为无线网络管理和自组织网络等。" ]
  [ "周凡钦（1988- ），男，河南渑池人，博士，北京邮电大学在站博士后，主要研究方向为无线异构网络的资源管理和负载均衡等。" ]
  [ "丰雷（1987- ），男，北京人，北京邮电大学副教授、硕士生导师，主要研究方向为无线网络和智能电网的资源管理等。" ]
  [ "付澍（1985- ），男，贵州贵阳人，博士，重庆大学副教授、硕士生导师，主要研究方向为星地通信、NOMA、物联网、网络一体化等。" ]
  [ "邱雪松（1973- ），男，江西上饶人，博士，北京邮电大学教授、博士生导师，主要研究方向为网络管理与通信软件等。" ]
- 基金信息：
  
  国家重点研发计划基金资助项目(2018YFE0205502)
- DOI：10.11959/j.issn.1000-436X.2020218
  中图分类号： TN929.5
- 网络出版日期：2020-12，
  
  纸质出版日期：2020-12-25
- 稿件说明：
移动端阅览
喻鹏, 张俊也, 李文璟, 等. 移动边缘网络中基于双深度Q学习的高能效资源分配方法[J]. 通信学报, 2020,41(12):148-161.

Peng YU, Junye ZHANG, Wenjing LI, et al. Energy-efficient resource allocation method in mobile edge network based on double deep Q-learning[J]. Journal on communications, 2020, 41(12): 148-161.
喻鹏, 张俊也, 李文璟, 等. 移动边缘网络中基于双深度Q学习的高能效资源分配方法[J]. 通信学报, 2020,41(12):148-161. DOI： 10.11959/j.issn.1000-436X.2020218.

Peng YU, Junye ZHANG, Wenjing LI, et al. Energy-efficient resource allocation method in mobile edge network based on double deep Q-learning[J]. Journal on communications, 2020, 41(12): 148-161. DOI： 10.11959/j.issn.1000-436X.2020218.

摘要

为了提升移动边缘网络中系统的能量使用效率，面向多任务、多终端设备、多边缘网关、多边缘服务器共存网络架构的下行通信过程，提出了一种基于双深度Q学习（DDQL）的通信、计算、存储融合资源分配方法。以任务平均能耗最小化为优化目标，设置任务时延和通信、计算、存储资源限制等约束条件，构建了对应的资源分配模型。依据模型特征，基于DDQL框架，提出了适用于通信和计算资源智能决策、存储资源按需分配的资源分配模型和算法。仿真结果表明，所提出的基于DDQL资源分配方法可以有效地解决多任务资源分配问题，具有较好的收敛性和较低的时间复杂度，在保障业务服务质量的同时，相对于基于随机算法、贪心算法、粒子群优化算法、深度Q学习等方法，降低了至少5%的任务平均能耗。

Abstract

To improve the system energy efficiency in mobile edge networks

a resource allocation method based on double deep Q-learning(DDQL) for integration of communication

computing

storage resources was proposed for the downlink communication process under the network architecture of multiple tasks

end devices

edge gateways and edge servers.A resource allocation model was constructed

which took the minimization of average energy consumption of tasks as the optimization goal and set the constraints of task delay limits and communication

computing

and storage resource limits.According to the model characteristics

a suitable resource allocation model and method based on DDQL framework was proposed to make intelligent allocation decisions for communication and computing resources and allocate storage resources on demand.Simulation results show that the proposed DDQL-based solution can effectively solve the multi-task resource allocation problem with good converge and low time complexity

and it reduces the average energy consumption of tasks by at least 5% compared with the solving methods based on random algorithm

greedy algorithm

particle swarm optimization algorithm and deep Q-learning while ensuring the quality of service.

关键词

Keywords

references

陈亮 , 余少华 . 6G 移动通信发展趋势初探（特邀） [J ] . 光通信研究 , 2019 , 45 ( 4 ): 1 - 8 .

CHEN L , YU S H . Preliminary Study on the Trend of 6G Mobile Communication (special invitation) [J ] . Study on Optical Communications , 2019 , 45 ( 4 ): 1 - 8 .

田辉 , 范绍帅 , 吕昕晨 , 等 . 面向 5G 需求的移动边缘计算 [J ] . 北京邮电大学学报 , 2017 , 40 ( 2 ): 1 - 10 .

TIAN H , FAN S S , LYU X C , et al . Mobile edge computing for 5G requirements [J ] . Journal of Beijing University of Posts and Telecommunications , 2017 , 40 ( 2 ): 1 - 10 .

STRINATI E C , BARBAROSSA S , GONZALEZ-JIMENEZ J L , et al . 6G:the next frontier:from holographic messaging to artificial intelligence using subterahertz and visible light communication [J ] . IEEE Vehicular Technology Magazine , 2019 , 14 ( 3 ): 42 - 50 .

毕成 . 5G无线网络节能技术研究 [J ] . 电信工程技术与标准化 , 2020 , 33 ( 8 ): 1 - 5 .

BI C . Research on energy saving technology of 5G wireless network [J ] . Telecom Engineering Technics and Standardization , 2020 , 33 ( 8 ): 1 - 5 .

赵亚军 , 郁光辉 , 徐汉青 . 6G移动通信网络:愿景、挑战与关键技术 [J ] . 中国科学:信息科学 , 2019 , 8 ( 49 ): 963 - 987 .

ZHAO Y J , YU G H , XU H Q . 6G mobile communication networks:vision,challenges,and key technologies [J ] . Scientia Sinica (Informationis) , 2019 , 8 ( 49 ): 963 - 987 .

CAI S , CHE Y , DUAN J , et al . Green 5G heterogeneous networks through dynamic small-cell operation [J ] . IEEE Journal on Selected Areas in Communications , 2016 , 34 ( 5 ): 1103 - 1115 .

ZHOU Y , TIAN L , LIU L , et al . Fog computing enabled future mobile communication networks:a convergence of communication and computing [J ] . IEEE Communications Magazine , 2019 , 57 ( 5 ): 20 - 27 .

LIU L , ZHOU Y , YUAN J , et al . Economically optimal MS association for multimedia content delivery in cache-enabled heterogeneous cloud radio access networks [J ] . IEEE Journal on Selected Areas in Communications , 2019 , 37 ( 7 ): 1584 - 1593 .

BI S , HUANG L , ZHANG Y J A . Joint optimization of service caching placement and computation offloading in mobile edge computing systems [J ] . IEEE Transactions on Wireless Communications , 2020 , 19 ( 7 ): 4947 - 4963 .

MAO Y , ZHANG J , LETAIEF K B . Dynamic computation offloading for mobile-edge computing with energy harvesting devices [J ] . IEEE Journal on Selected Areas in Communications , 2016 , 34 ( 12 ): 3590 - 3605 .

LIU L , ZHOU Y , GARCIA V , et al . Load aware joint CoMP clustering and inter-cell resource scheduling in heterogeneous ultra dense cellular networks [J ] . IEEE Transactions on Vehicular Technology , 2017 , 67 ( 3 ): 2741 - 2755 .

BADRI H , BAHREINI T , GROSU D , et al . Risk-based optimization of resource provisioning in mobile edge computing [C ] // 2018 IEEE/ACM Symposium on Edge Computing (SEC) . Piscataway:IEEE Press , 2018 : 328 - 330 .

WANG L , JIAO L , LI J , et al . Moera:mobility-agnostic online resource allocation for edge computing [J ] . IEEE Transactions on Mobile Computing , 2019 , 18 ( 8 ): 1843 - 1856 .

CHEN X , JIAO L , LI W , et al . Efficient multi-user computation offloading for mobile-edge cloud computing [J ] . IEEE/ACM Transactions on Networking , 2015 , 24 ( 5 ): 2795 - 2808 .

ZENG D , GU L , PAN S , et al . Resource management at the network edge:a deep reinforcement learning approach [J ] . IEEE Network , 2019 , 33 ( 3 ): 26 - 33 .

梁应敞 , 谭俊杰 , NIYATO D . 智能无线通信技术研究概况 [J ] . 通信学报 , 2020 , 41 ( 7 ): 1 - 17 .

LIANG Y C , TAN J J , NIYATO D . Overview on intelligent wireless communication technology [J ] . Journal on Communications , 2020 , 41 ( 7 ): 1 - 17 .

XU J , CHEN L , REN S . Online learning for offloading and autoscaling in energy harvesting mobile edge computing [J ] . IEEE Transactions on Cognitive Communications and Networking , 2017 , 3 ( 3 ): 361 - 373 .

ZHANG C , PATRAS P , HADDADI H . Deep learning in mobile and wireless networking:a survey [J ] . IEEE Communications Surveys ＆Tutorials , 2019 , 21 ( 3 ): 2224 - 2287 .

ZHAO N , LIANG Y C , NIYATO D , et al . Deep reinforcement learning for user association and resource allocation in heterogeneous cellular networks [J ] . IEEE Transactions on Wireless Communications , 2019 , 18 ( 11 ): 5141 - 5152 .

LI J , GAO H , LYU T , et al . Deep reinforcement learning based computation offloading and resource allocation for MEC [C ] // 2018 IEEE Wireless Communications and Networking Conference . Piscataway:IEEE Press , 2018 : 1 - 6 .

WANG J , ZHAO L , LIU J , et al . Smart resource allocation for mobile edge computing:a deep reinforcement learning approach [J ] . IEEE Transactions on Emerging Topics in Computing , 2019 ,,doi:10.1109/TETC.2019.2902661.

LI Y , QI F , WANG Z , et al . Distributed edge computing offloading algorithm based on deep reinforcement learning [J ] . IEEE Access , 2020 , 8 : 85204 - 85215 .

VAN H H , GUEZ A , SILVER D . Deep reinforcement learning with double Q-learning [C ] // Proceedings of the AAAI Conference on Artificial Intelligence . Palo Alto:AAAI Press , 2016 : 2094 - 2100 .

CHEN X , ZHANG H , WU C , et al . Optimized computation of-floading performance in virtual edge computing systems via deep reinforcement learning [J ] . IEEE Internet of Things Journal , 2018 , 6 ( 3 ): 4005 - 4018 .

MAO Y , YOU C , ZHANG J , et al . A survey on mobile edge computing:the communication perspective [J ] . IEEE Communications Surveys ＆ Tutorials , 2017 , 19 ( 4 ): 2322 - 2358 .

刘全 , 翟建伟 , 章宗长 , 等 . 深度强化学习综述 [J ] . 计算机学报 , 2018 , 41 ( 1 ): 1 - 27 .

LIU Q , ZHAI J W , ZHANG Z Z , et al . A survey on deep reinforcement learning [J ] . Chinese Journal of Computers , 2018 , 41 ( 1 ): 1 - 27 .

MNIH V , KAVUKCUOGLU K , SILVER D , et al . Human-level control through deep reinforcement learning [J ] . Nature , 2015 , 518 ( 7540 ): 529 - 533 .

CAO X , WANG F , XU J , et al . Joint computation and communication cooperation for energy-efficient mobile edge computing [J ] . IEEE Internet of Things Journal , 2018 , 6 ( 3 ): 4188 - 4200 .

TRAN T X , POMPILI D . Joint task offloading and resource allocation for multi-server mobile-edge computing networks [J ] . IEEE Transactions on Vehicular Technology , 2018 , 68 ( 1 ): 856 - 868 .

YANG X , YU P , FENG L , et al . A deep reinforcement learning based mechanism for cell outage compensation in 5G UDN [C ] // 2019 IFIP/IEEE Symposium on Integrated Network and Service Management . Piscataway:IEEE Press , 2019 : 476 - 481 .

WANG S , PAN C , YIN C . Joint heterogeneous tasks offloading and resource allocation in mobile edge computing systems [C ] // 2018 10th International Conference on Wireless Communications and Signal Processing . Piscataway:IEEE Press , 2018 : 1 - 6 .

ARULKUMARAN K , DEISENROTH M P , BRUNDAGE M , et al . Deep reinforcement learning:a brief survey [J ] . IEEE Signal Processing Magazine , 2017 , 34 ( 6 ): 26 - 38 .

浏览量

530

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

D2D协作边缘缓存系统中基于传输时延的缓存策略

无线传感器网络自适应目标跟踪节点调度算法