Energy-efficient resource allocation method in mobile edge network based on double deep Q-learning

Peng YU; Junye ZHANG; Wenjing LI; Fanqin ZHOU; Lei FENG; Shu FU; Xuesong QIU

doi:10.11959/j.issn.1000-436X.2020218

您当前的位置：

首页 >

文章列表页 >

Energy-efficient resource allocation method in mobile edge network based on double deep Q-learning

Papers | 更新时间：2024-06-05

- Energy-efficient resource allocation method in mobile edge network based on double deep Q-learning
- Journal on Communications Vol. 41, Issue 12, Pages: 148-161(2020)
- 作者机构：
  
  1. 北京邮电大学网络与交换技术国家重点实验室，北京 100876
  2. 重庆大学微电子与通信工程学院，重庆 400044
- 作者简介：
- 基金信息：
  
  The National Key Research and Development Program of China(2018YFE0205502)
- DOI：10.11959/j.issn.1000-436X.2020218
  CLC： TN929.5
- Online First：2020-12，
  
  Published：25 December 2020
- 稿件说明：
移动端阅览
Peng YU, Junye ZHANG, Wenjing LI, et al. Energy-efficient resource allocation method in mobile edge network based on double deep Q-learning[J]. Journal on Communications, 2020, 41(12): 148-161.
DOI：

Peng YU, Junye ZHANG, Wenjing LI, et al. Energy-efficient resource allocation method in mobile edge network based on double deep Q-learning[J]. Journal on Communications, 2020, 41(12): 148-161. DOI： 10.11959/j.issn.1000-436X.2020218.

摘要

为了提升移动边缘网络中系统的能量使用效率，面向多任务、多终端设备、多边缘网关、多边缘服务器共存网络架构的下行通信过程，提出了一种基于双深度Q学习（DDQL）的通信、计算、存储融合资源分配方法。以任务平均能耗最小化为优化目标，设置任务时延和通信、计算、存储资源限制等约束条件，构建了对应的资源分配模型。依据模型特征，基于DDQL框架，提出了适用于通信和计算资源智能决策、存储资源按需分配的资源分配模型和算法。仿真结果表明，所提出的基于DDQL资源分配方法可以有效地解决多任务资源分配问题，具有较好的收敛性和较低的时间复杂度，在保障业务服务质量的同时，相对于基于随机算法、贪心算法、粒子群优化算法、深度Q学习等方法，降低了至少5%的任务平均能耗。

Abstract

To improve the system energy efficiency in mobile edge networks

a resource allocation method based on double deep Q-learning(DDQL) for integration of communication

computing

storage resources was proposed for the downlink communication process under the network architecture of multiple tasks

end devices

edge gateways and edge servers.A resource allocation model was constructed

which took the minimization of average energy consumption of tasks as the optimization goal and set the constraints of task delay limits and communication

computing

and storage resource limits.According to the model characteristics

a suitable resource allocation model and method based on DDQL framework was proposed to make intelligent allocation decisions for communication and computing resources and allocate storage resources on demand.Simulation results show that the proposed DDQL-based solution can effectively solve the multi-task resource allocation problem with good converge and low time complexity

and it reduces the average energy consumption of tasks by at least 5% compared with the solving methods based on random algorithm

greedy algorithm

particle swarm optimization algorithm and deep Q-learning while ensuring the quality of service.

关键词

Keywords

references

陈亮 , 余少华 . 6G 移动通信发展趋势初探（特邀） [J ] . 光通信研究 , 2019 , 45 ( 4 ): 1 - 8 .

CHEN L , YU S H . Preliminary Study on the Trend of 6G Mobile Communication (special invitation) [J ] . Study on Optical Communications , 2019 , 45 ( 4 ): 1 - 8 .

田辉 , 范绍帅 , 吕昕晨 , 等 . 面向 5G 需求的移动边缘计算 [J ] . 北京邮电大学学报 , 2017 , 40 ( 2 ): 1 - 10 .

TIAN H , FAN S S , LYU X C , et al . Mobile edge computing for 5G requirements [J ] . Journal of Beijing University of Posts and Telecommunications , 2017 , 40 ( 2 ): 1 - 10 .

STRINATI E C , BARBAROSSA S , GONZALEZ-JIMENEZ J L , et al . 6G:the next frontier:from holographic messaging to artificial intelligence using subterahertz and visible light communication [J ] . IEEE Vehicular Technology Magazine , 2019 , 14 ( 3 ): 42 - 50 .

毕成 . 5G无线网络节能技术研究 [J ] . 电信工程技术与标准化 , 2020 , 33 ( 8 ): 1 - 5 .

BI C . Research on energy saving technology of 5G wireless network [J ] . Telecom Engineering Technics and Standardization , 2020 , 33 ( 8 ): 1 - 5 .

赵亚军 , 郁光辉 , 徐汉青 . 6G移动通信网络:愿景、挑战与关键技术 [J ] . 中国科学:信息科学 , 2019 , 8 ( 49 ): 963 - 987 .

ZHAO Y J , YU G H , XU H Q . 6G mobile communication networks:vision,challenges,and key technologies [J ] . Scientia Sinica (Informationis) , 2019 , 8 ( 49 ): 963 - 987 .

CAI S , CHE Y , DUAN J , et al . Green 5G heterogeneous networks through dynamic small-cell operation [J ] . IEEE Journal on Selected Areas in Communications , 2016 , 34 ( 5 ): 1103 - 1115 .

ZHOU Y , TIAN L , LIU L , et al . Fog computing enabled future mobile communication networks:a convergence of communication and computing [J ] . IEEE Communications Magazine , 2019 , 57 ( 5 ): 20 - 27 .

LIU L , ZHOU Y , YUAN J , et al . Economically optimal MS association for multimedia content delivery in cache-enabled heterogeneous cloud radio access networks [J ] . IEEE Journal on Selected Areas in Communications , 2019 , 37 ( 7 ): 1584 - 1593 .

BI S , HUANG L , ZHANG Y J A . Joint optimization of service caching placement and computation offloading in mobile edge computing systems [J ] . IEEE Transactions on Wireless Communications , 2020 , 19 ( 7 ): 4947 - 4963 .

MAO Y , ZHANG J , LETAIEF K B . Dynamic computation offloading for mobile-edge computing with energy harvesting devices [J ] . IEEE Journal on Selected Areas in Communications , 2016 , 34 ( 12 ): 3590 - 3605 .

LIU L , ZHOU Y , GARCIA V , et al . Load aware joint CoMP clustering and inter-cell resource scheduling in heterogeneous ultra dense cellular networks [J ] . IEEE Transactions on Vehicular Technology , 2017 , 67 ( 3 ): 2741 - 2755 .

BADRI H , BAHREINI T , GROSU D , et al . Risk-based optimization of resource provisioning in mobile edge computing [C ] // 2018 IEEE/ACM Symposium on Edge Computing (SEC) . Piscataway:IEEE Press , 2018 : 328 - 330 .

WANG L , JIAO L , LI J , et al . Moera:mobility-agnostic online resource allocation for edge computing [J ] . IEEE Transactions on Mobile Computing , 2019 , 18 ( 8 ): 1843 - 1856 .

CHEN X , JIAO L , LI W , et al . Efficient multi-user computation offloading for mobile-edge cloud computing [J ] . IEEE/ACM Transactions on Networking , 2015 , 24 ( 5 ): 2795 - 2808 .

ZENG D , GU L , PAN S , et al . Resource management at the network edge:a deep reinforcement learning approach [J ] . IEEE Network , 2019 , 33 ( 3 ): 26 - 33 .

梁应敞 , 谭俊杰 , NIYATO D . 智能无线通信技术研究概况 [J ] . 通信学报 , 2020 , 41 ( 7 ): 1 - 17 .

LIANG Y C , TAN J J , NIYATO D . Overview on intelligent wireless communication technology [J ] . Journal on Communications , 2020 , 41 ( 7 ): 1 - 17 .

XU J , CHEN L , REN S . Online learning for offloading and autoscaling in energy harvesting mobile edge computing [J ] . IEEE Transactions on Cognitive Communications and Networking , 2017 , 3 ( 3 ): 361 - 373 .

ZHANG C , PATRAS P , HADDADI H . Deep learning in mobile and wireless networking:a survey [J ] . IEEE Communications Surveys ＆Tutorials , 2019 , 21 ( 3 ): 2224 - 2287 .

ZHAO N , LIANG Y C , NIYATO D , et al . Deep reinforcement learning for user association and resource allocation in heterogeneous cellular networks [J ] . IEEE Transactions on Wireless Communications , 2019 , 18 ( 11 ): 5141 - 5152 .

LI J , GAO H , LYU T , et al . Deep reinforcement learning based computation offloading and resource allocation for MEC [C ] // 2018 IEEE Wireless Communications and Networking Conference . Piscataway:IEEE Press , 2018 : 1 - 6 .

WANG J , ZHAO L , LIU J , et al . Smart resource allocation for mobile edge computing:a deep reinforcement learning approach [J ] . IEEE Transactions on Emerging Topics in Computing , 2019 ,,doi:10.1109/TETC.2019.2902661.

LI Y , QI F , WANG Z , et al . Distributed edge computing offloading algorithm based on deep reinforcement learning [J ] . IEEE Access , 2020 , 8 : 85204 - 85215 .

VAN H H , GUEZ A , SILVER D . Deep reinforcement learning with double Q-learning [C ] // Proceedings of the AAAI Conference on Artificial Intelligence . Palo Alto:AAAI Press , 2016 : 2094 - 2100 .

CHEN X , ZHANG H , WU C , et al . Optimized computation of-floading performance in virtual edge computing systems via deep reinforcement learning [J ] . IEEE Internet of Things Journal , 2018 , 6 ( 3 ): 4005 - 4018 .

MAO Y , YOU C , ZHANG J , et al . A survey on mobile edge computing:the communication perspective [J ] . IEEE Communications Surveys ＆ Tutorials , 2017 , 19 ( 4 ): 2322 - 2358 .

刘全 , 翟建伟 , 章宗长 , 等 . 深度强化学习综述 [J ] . 计算机学报 , 2018 , 41 ( 1 ): 1 - 27 .

LIU Q , ZHAI J W , ZHANG Z Z , et al . A survey on deep reinforcement learning [J ] . Chinese Journal of Computers , 2018 , 41 ( 1 ): 1 - 27 .

MNIH V , KAVUKCUOGLU K , SILVER D , et al . Human-level control through deep reinforcement learning [J ] . Nature , 2015 , 518 ( 7540 ): 529 - 533 .

CAO X , WANG F , XU J , et al . Joint computation and communication cooperation for energy-efficient mobile edge computing [J ] . IEEE Internet of Things Journal , 2018 , 6 ( 3 ): 4188 - 4200 .

TRAN T X , POMPILI D . Joint task offloading and resource allocation for multi-server mobile-edge computing networks [J ] . IEEE Transactions on Vehicular Technology , 2018 , 68 ( 1 ): 856 - 868 .

YANG X , YU P , FENG L , et al . A deep reinforcement learning based mechanism for cell outage compensation in 5G UDN [C ] // 2019 IFIP/IEEE Symposium on Integrated Network and Service Management . Piscataway:IEEE Press , 2019 : 476 - 481 .

WANG S , PAN C , YIN C . Joint heterogeneous tasks offloading and resource allocation in mobile edge computing systems [C ] // 2018 10th International Conference on Wireless Communications and Signal Processing . Piscataway:IEEE Press , 2018 : 1 - 6 .

ARULKUMARAN K , DEISENROTH M P , BRUNDAGE M , et al . Deep reinforcement learning:a brief survey [J ] . IEEE Signal Processing Magazine , 2017 , 34 ( 6 ): 26 - 38 .

Views

2169

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Caching strategy based on transmission delay for D2D cooperative edge caching system

Adaptive node scheduling algorithm for target tracking in wireless sensor networks

Related Author

Yan CAI

Fan WU

Hongbo ZHU

Xu LU

Liang-lun CHENG

Shi-liang LUO

Related Institution

Key Laboratory of Wireless Communications of Jiangsu Province, Nanjing University of Posts and Telecommunications

College of Automatization，Guangdong University of Technology

College of Automatization，Guangdong Polytechnic Normal University

College of Automatization，Guangdong University of Technology，Guangzhou 510006，China;2.College of Automatization，Guangdong Polytechnic Normal University

AI问答

⁰