Deep reinforcement learning based task allocation mechanism for intelligent inspection in energy Internet

Siya XU; Yifei XING; Shaoyong GUO; Chao YANG; Xuesong QIU; Luoming MENG

doi:10.11959/j.issn.1000-436x.2021071

您当前的位置：

首页 >

文章列表页 >

Deep reinforcement learning based task allocation mechanism for intelligent inspection in energy Internet

Correspondences | 更新时间：2024-06-06

- Deep reinforcement learning based task allocation mechanism for intelligent inspection in energy Internet
- Journal on Communications Vol. 42, Issue 5, Pages: 191-204(2021)
- 作者机构：
  
  1. 北京邮电大学网络技术与交换重点实验室，北京 100876
  2. 国网辽宁省电力有限公司信息通信分公司，辽宁沈阳 110004
- 作者简介：
- 基金信息：
  
  The National Key Research and Development Program of China(2019YFB2102302);The National Natural Science Foundation of China(61702048);Innovative Development Project of Industrial Internet (Construction of Industrial In-ternet Test Bed Based on Ubiquitous Power Internet of Things)
- DOI：10.11959/j.issn.1000-436x.2021071
  CLC： TN92
- Online First：2021-05，
  
  Published：25 May 2021
- 稿件说明：
移动端阅览
Siya XU, Yifei XING, Shaoyong GUO, et al. Deep reinforcement learning based task allocation mechanism for intelligent inspection in energy Internet[J]. Journal on Communications, 2021, 42(5): 191-204.
DOI：

Siya XU, Yifei XING, Shaoyong GUO, et al. Deep reinforcement learning based task allocation mechanism for intelligent inspection in energy Internet[J]. Journal on Communications, 2021, 42(5): 191-204. DOI： 10.11959/j.issn.1000-436x.2021071.

摘要

在能源互联网中引入无人机进行电力线路巡查，并借助移动边缘计算技术实现巡检任务的接入和处理，可降低服务成本，提高工作效率。但是，由于无人机数据传输需求和地理位置的动态变化，易造成边缘服务器负载不均衡，致使巡检业务处理时延和网络能耗较高。为解决以上问题，提出基于深度强化学习的能源互联网智能巡检任务分配机制。首先，综合考虑无人机和边缘节点的运动轨迹、业务差异化的服务需求、边缘节点有限的服务能力等，建立面向时延、能耗等多目标联合优化的双层边缘网络任务卸载模型。进而，基于 Lyapunov 优化理论和双时间尺度机制，采用近端策略优化的深度强化学习算法，对固定边缘汇聚层和移动边缘接入层边缘节点间的连接关系和卸载策略进行求解。仿真结果表明，所提机制能够在保证系统稳定的情况下降低服务时延和系统能耗。

Abstract

In order to reduce the cost and improve efficiency of power line inspection

UAV (unmanned aerial vehicle)

which use mobile edge computing technology to access and process service data

are used to inspect power lines in the energy internet.However

due to the dynamic changes of UAV data transmission demand and geographical location

the edge server load will be unbalanced

which causes higher service processing delay and network energy consumption.Thus

an intelligent inspection task allocation mechanism for energy internet based on deep reinforcement learning was proposed.First

a two-layer edge network task offloading model was established to archive joint optimization of multi-objectives

such as delay and energy consumption.It was designed by comprehensively considering the route of UAV and edge nodes

different demands of services and limited service capabilities of edge nodes.Furthermore

based on Lyapunov optimization theory and dual-time-scaled mechanism

proximal policy optimization algorithm based deep reinforcement learning was used to solve the connection relationship and offloading strategy of edge servers between fixed edge sink layer and mobile edge access layer.The simulation results show that

the proposed mechanism can reduce the service request delay and system energy consumption while ensuring the stability of system.

关键词

Keywords

references

李君海 , 张苗苗 , 熊道洋 . 基于实时信息传输技术的无人机巡检管控平台 [J ] . 测绘与空间地理信息 , 2020 , 43 ( 6 ): 165 - 167 , 171 .

LI J H , ZHANG M M , XIONG D Y . An UAV power inspection management platform based on real-time message transmission technology [J ] . Geomatics ＆ Spatial Information Technology , 2020 , 43 ( 6 ): 165 - 167 , 171 .

陈兰波 . 电力线路无人机巡检方案研究 [J ] . 科技与创新 , 2020 ( 11 ): 36 - 38 , 41 .

CHEN L B . Research on UAV patrol scheme of power line [J ] . Science and Technology ＆ Innovation , 2020 ( 11 ): 36 - 38 , 41 .

严波 , 林世忠 , 张振威 , 等 . 无人机电力巡检技术应用分析 [J ] . 自动化应用 , 2019 ( 12 ): 155 - 156 .

YAN B , LIN S Z , ZHANG Z W , et al . Application analysis of UAV power inspection technology [J ] . Automation Application , 2019 ( 12 ): 155 - 156 .

HAN D S , LI S J , CHEN Z X . Hybrid energy ratio allocation algorithm in a multi-base-station collaboration system [J ] . IEEE Access , 2019 , 7 : 147001 - 147009 .

JEONG S , SIMEONE O , KANG J . Mobile edge computing via a UAV-mounted cloudlet:optimization of bit allocation and path planning [J ] . IEEE Transactions on Vehicular Technology , 2018 , 67 ( 3 ): 2049 - 2063 .

WU Q Q , ZENG Y , ZHANG R . Joint trajectory and communication design for multi-UAV enabled wireless networks [J ] . IEEE Transactions on Wireless Communications , 2018 , 17 ( 3 ): 2109 - 2121 .

WEI X L , TANG C G , FAN J H , et al . Joint optimization of energy consumption and delay in cloud-to-thing continuum [J ] . IEEE Internet of Things Journal , 2019 , 6 ( 2 ): 2325 - 2337 .

HU X Y , WONG K K , YANG K , et al . UAV-assisted relaying and edge computing:scheduling and trajectory optimization [J ] . IEEE Transactions on Wireless Communications , 2019 , 18 ( 10 ): 4738 - 4752 .

HUA M , WANG Y , ZHANG Z M , et al . Power-efficient communication in UAV-aided wireless sensor networks [J ] . IEEE Communications Letters , 2018 , 22 ( 6 ): 1264 - 1267 .

ZHANG T K , XU Y , LOO J , et al . Joint computation and communication design for UAV-assisted mobile edge computing in IoT [J ] . IEEE Transactions on Industrial Informatics , 2020 , 16 ( 8 ): 5505 - 5516 .

ATEYA A A , MUTHANNA A , KIRICHEK R , et al . Energy-and latency-aware hybrid offloading algorithm for UAVs [J ] . IEEE Access , 2019 , 7 : 37587 - 37600 .

王立科 . 无人机技术在电力巡检信息化管理中的应用研究 [J ] . 机电信息 , 2019 ( 27 ): 88 - 89 .

WANG L K . Research on application of UAV technology in information management of power patrol inspection [J ] . Mechanical and Electrical Information , 2019 ( 27 ): 88 - 89 .

马青岷 . 无人机电力巡检及三维模型重建技术研究 [D ] . 济南:山东大学 , 2017 .

MA Q M . Research of powerline inspection based on UAV and 3D model technology [D ] . Jinan:Shandong University , 2017 .

SHUO W , XING Z , ZHI Y , et al . Cooperative edge computing with sleep control under nonuniform traffic in mobile edge networks [J ] . IEEE Internet of Things Journal , 2019 , 6 ( 3 ): 4295 - 4306 .

SHAW J A . Radiometry and the Friis transmission equation [J ] . American Journal of Physics , 2013 , 81 ( 1 ): 33 - 37 .

DOWNEY C . Understanding wireless range calculations [J ] . Electronic Design , 2013 .

MAO Y Y , ZHANG J , SONG S H , et al . Power-delay tradeoff in multi-user mobile-edge computing systems [C ] // 2016 IEEE Global Communications Conference . Piscataway:IEEE Press , 2016 : 1 - 6 .

BURD T D , BRODERSEN R W . Processor design for portable systems [J ] . Journal of VLSI Signal Processing Systems for Signal,Image and Video Technology , 1996 , 13 ( 2/3 ): 203 - 221 .

FAN X B , WEBER W D , BARROSO L A . Power provisioning for a warehouse-sized computer [J ] . ACM SIGARCH Computer Architecture News , 2007 , 35 ( 2 ): 13 - 23 .

CHEN L X , ZHOU S , XU J . Computation peer offloading for energy-constrained mobile edge computing in small-cell networks [J ] . IEEE/ACM Transactions on Networking , 2018 , 26 ( 4 ): 1619 - 1632 .

YANG L , CAO J N , TANG S J , et al . A framework for partitioning and execution of data stream applications in mobile cloud computing [C ] // 2012 IEEE Fifth International Conference on Cloud Computing . Piscataway:IEEE Press , 2012 : 794 - 802 .

CHEN L X , XU J , ZHOU S . Computation peer offloading in mobile edge computing with energy budgets [C ] // GLOBECOM 2017 - 2017 IEEE Global Communications Conference . Piscataway:IEEE Press , 2017 : 1 - 6 .

NEELY J M . Stochastic network optimization with application to communication and queueing systems [J ] . Synthesis Lectures on Communication Networks , 2010 , 3 ( 1 ): 211 .

XU S Y , LIU Q C , GONG B , et al . RJCC:reinforcement-learningbased joint communicational-and-computational resource allocation mechanism for smart city IoT [J ] . IEEE Internet of Things Journal , 2020 , 7 ( 9 ): 8059 - 8076 .

HEESS N , DHRUVA T B , SRIRAM S , et al . Emergence of locomotion behaviours in rich environments [J ] . arXiv Preprint,arXiv:1707.02286 , 2017 .

SCHULMAN J , WOLSKI F , DHARIWAL P , et al . Proximal policy optimization algorithms [J ] . arXiv Preprint,arXiv:1707.06347 , 2017 .

王志夫 . 基于深度强化学习的双足机器人步行运动控制 [D ] . 济南:山东大学 , 2020 .

WANG Z F . Deep reinforcement learning based walking control of biped robot [D ] . Jinan:Shandong University , 2020 .

王鸿涛 . 基于强化学习的机械臂自学习控制 [D ] . 哈尔滨:哈尔滨工业大学 , 2019 .

WANG H T . Self learning control of mechanical arm based on reinforcement learning [D ] . Harbin:Harbin Institute of Technology , 2019 .

胡彦娟 . 移动边缘计算中任务卸载及资源分配算法研究 [D ] . 重庆:重庆邮电大学 , 2020 .

HU Y J . Research on task offloading and resource allocation algorithm in mobile edge computing [D ] . Chongqing:Chongqing University of Posts and Telecommunications , 2020 .

GAO Z B , WEN B , HUANG L F , et al . Q-learning-based power control for LTE enterprise femtocell networks [J ] . IEEE Systems Journal , 2017 , 11 ( 4 ): 2699 - 2707 .

LI R , ZHANG C Y , PATRAS P , et al . Learning driven mobility control of airborne base stations in emergency networks [J ] . ACM SIGMETRICS Performance Evaluation Review , 2019 , 46 ( 3 ): 163 - 166 .

KWAK J , KIM Y , LEE J , et al . DREAM:dynamic resource and task allocation for energy minimization in mobile cloud systems [J ] . IEEE Journal on Selected Areas in Communications , 2015 , 33 ( 12 ): 2510 - 2523 .

Views

2087

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Temporal accumulation-driven intelligent spectrum sensing method under low-SNR conditions

Multi-agent cooperative confrontation with proximal policy optimization in urban environments

Research on knowledge graph construction technology for cyber threat intelligence based on large language models

Research on task offloading and resource allocation in edge computing network of RIS assisted UAV based on Lyapunov

Intelligent anti-jamming decision algorithm based on proximal policy optimization

Related Author

Yang Xiaoniu

Qi Peihan

Zheng Shilian

Zhang Luxin

ZHUO Li

ZHANG Jing

ZHANG Hui

MI Guangming

Related Institution

School of Telecommunications Engineering, Xidian University

National Key Laboratory of Electromagnetic Space Security

Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology

School of Information Science and Technology, Beijing University of Technology

Computing Center, Peking University

AI问答

⁰