基于深度强化学习的能源互联网智能巡检任务分配机制

徐思雅; 邢逸斐; 郭少勇; 杨超; 邱雪松; 孟洛明

doi:10.11959/j.issn.1000-436x.2021071

您当前的位置：

首页 >

文章列表页 >

基于深度强化学习的能源互联网智能巡检任务分配机制

学术通信 | 更新时间：2024-06-06

- 基于深度强化学习的能源互联网智能巡检任务分配机制
- Deep reinforcement learning based task allocation mechanism for intelligent inspection in energy Internet
- 通信学报 2021年42卷第5期页码：191-204
- 作者机构：
  
  1. 北京邮电大学网络技术与交换重点实验室，北京 100876
  2. 国网辽宁省电力有限公司信息通信分公司，辽宁沈阳 110004
- 作者简介：
  
  [ "徐思雅（1988- ），女，北京人，博士，北京邮电大学讲师，主要研究方向为信息通信网络管理、SDN/NFV、移动边缘计算和人工智能等" ]
  [ "邢逸斐（1997- ），男，河北保定人，北京邮电大学硕士生，主要研究方向为智能电网网络管理、移动边缘计算等" ]
  [ "郭少勇（1985- ），男，河北邢台人，博士，北京邮电大学副教授，主要研究方向为区块链、物联网等" ]
  [ "杨超（1988- ），男，山东平度人，博士，国网辽宁省电力有限公司工程师，主要研究方向为电力物联网、人工智能工程化应用等" ]
  [ "邱雪松（1973- ），男，江西上饶人，博士，北京邮电大学教授、博士生导师，主要研究方向为网络与业务管理、物联网与区块链" ]
  [ "孟洛明（1955- ），男，河南洛阳人，博士，北京邮电大学教授、博士生导师，主要研究方向为通信网、网络管理、通信软件等" ]
- 基金信息：
  
  国家重点研发计划基金资助项目(2019YFB2102302);国家自然科学基金资助项目(61702048);工业互联网创新发展工程基金资助项目（基于泛在电力物联网的工业互联网测试床）
- DOI：10.11959/j.issn.1000-436x.2021071
  中图分类号： TN92
- 网络首发：2021-05，
  
  纸质出版：2021-05-25
- 稿件说明：
移动端阅览
徐思雅, 邢逸斐, 郭少勇, 等. 基于深度强化学习的能源互联网智能巡检任务分配机制[J]. 通信学报, 2021,42(5):191-204.

Siya XU, Yifei XING, Shaoyong GUO, et al. Deep reinforcement learning based task allocation mechanism for intelligent inspection in energy Internet[J]. Journal on Communications, 2021, 42(5): 191-204.
徐思雅, 邢逸斐, 郭少勇, 等. 基于深度强化学习的能源互联网智能巡检任务分配机制[J]. 通信学报, 2021,42(5):191-204. DOI： 10.11959/j.issn.1000-436x.2021071.

Siya XU, Yifei XING, Shaoyong GUO, et al. Deep reinforcement learning based task allocation mechanism for intelligent inspection in energy Internet[J]. Journal on Communications, 2021, 42(5): 191-204. DOI： 10.11959/j.issn.1000-436x.2021071.

摘要

在能源互联网中引入无人机进行电力线路巡查，并借助移动边缘计算技术实现巡检任务的接入和处理，可降低服务成本，提高工作效率。但是，由于无人机数据传输需求和地理位置的动态变化，易造成边缘服务器负载不均衡，致使巡检业务处理时延和网络能耗较高。为解决以上问题，提出基于深度强化学习的能源互联网智能巡检任务分配机制。首先，综合考虑无人机和边缘节点的运动轨迹、业务差异化的服务需求、边缘节点有限的服务能力等，建立面向时延、能耗等多目标联合优化的双层边缘网络任务卸载模型。进而，基于 Lyapunov 优化理论和双时间尺度机制，采用近端策略优化的深度强化学习算法，对固定边缘汇聚层和移动边缘接入层边缘节点间的连接关系和卸载策略进行求解。仿真结果表明，所提机制能够在保证系统稳定的情况下降低服务时延和系统能耗。

Abstract

In order to reduce the cost and improve efficiency of power line inspection

UAV (unmanned aerial vehicle)

which use mobile edge computing technology to access and process service data

are used to inspect power lines in the energy internet.However

due to the dynamic changes of UAV data transmission demand and geographical location

the edge server load will be unbalanced

which causes higher service processing delay and network energy consumption.Thus

an intelligent inspection task allocation mechanism for energy internet based on deep reinforcement learning was proposed.First

a two-layer edge network task offloading model was established to archive joint optimization of multi-objectives

such as delay and energy consumption.It was designed by comprehensively considering the route of UAV and edge nodes

different demands of services and limited service capabilities of edge nodes.Furthermore

based on Lyapunov optimization theory and dual-time-scaled mechanism

proximal policy optimization algorithm based deep reinforcement learning was used to solve the connection relationship and offloading strategy of edge servers between fixed edge sink layer and mobile edge access layer.The simulation results show that

the proposed mechanism can reduce the service request delay and system energy consumption while ensuring the stability of system.

关键词

Keywords

references

李君海 , 张苗苗 , 熊道洋 . 基于实时信息传输技术的无人机巡检管控平台 [J ] . 测绘与空间地理信息 , 2020 , 43 ( 6 ): 165 - 167 , 171 .

LI J H , ZHANG M M , XIONG D Y . An UAV power inspection management platform based on real-time message transmission technology [J ] . Geomatics ＆ Spatial Information Technology , 2020 , 43 ( 6 ): 165 - 167 , 171 .

陈兰波 . 电力线路无人机巡检方案研究 [J ] . 科技与创新 , 2020 ( 11 ): 36 - 38 , 41 .

CHEN L B . Research on UAV patrol scheme of power line [J ] . Science and Technology ＆ Innovation , 2020 ( 11 ): 36 - 38 , 41 .

严波 , 林世忠 , 张振威 , 等 . 无人机电力巡检技术应用分析 [J ] . 自动化应用 , 2019 ( 12 ): 155 - 156 .

YAN B , LIN S Z , ZHANG Z W , et al . Application analysis of UAV power inspection technology [J ] . Automation Application , 2019 ( 12 ): 155 - 156 .

HAN D S , LI S J , CHEN Z X . Hybrid energy ratio allocation algorithm in a multi-base-station collaboration system [J ] . IEEE Access , 2019 , 7 : 147001 - 147009 .

JEONG S , SIMEONE O , KANG J . Mobile edge computing via a UAV-mounted cloudlet:optimization of bit allocation and path planning [J ] . IEEE Transactions on Vehicular Technology , 2018 , 67 ( 3 ): 2049 - 2063 .

WU Q Q , ZENG Y , ZHANG R . Joint trajectory and communication design for multi-UAV enabled wireless networks [J ] . IEEE Transactions on Wireless Communications , 2018 , 17 ( 3 ): 2109 - 2121 .

WEI X L , TANG C G , FAN J H , et al . Joint optimization of energy consumption and delay in cloud-to-thing continuum [J ] . IEEE Internet of Things Journal , 2019 , 6 ( 2 ): 2325 - 2337 .

HU X Y , WONG K K , YANG K , et al . UAV-assisted relaying and edge computing:scheduling and trajectory optimization [J ] . IEEE Transactions on Wireless Communications , 2019 , 18 ( 10 ): 4738 - 4752 .

HUA M , WANG Y , ZHANG Z M , et al . Power-efficient communication in UAV-aided wireless sensor networks [J ] . IEEE Communications Letters , 2018 , 22 ( 6 ): 1264 - 1267 .

ZHANG T K , XU Y , LOO J , et al . Joint computation and communication design for UAV-assisted mobile edge computing in IoT [J ] . IEEE Transactions on Industrial Informatics , 2020 , 16 ( 8 ): 5505 - 5516 .

ATEYA A A , MUTHANNA A , KIRICHEK R , et al . Energy-and latency-aware hybrid offloading algorithm for UAVs [J ] . IEEE Access , 2019 , 7 : 37587 - 37600 .

王立科 . 无人机技术在电力巡检信息化管理中的应用研究 [J ] . 机电信息 , 2019 ( 27 ): 88 - 89 .

WANG L K . Research on application of UAV technology in information management of power patrol inspection [J ] . Mechanical and Electrical Information , 2019 ( 27 ): 88 - 89 .

马青岷 . 无人机电力巡检及三维模型重建技术研究 [D ] . 济南:山东大学 , 2017 .

MA Q M . Research of powerline inspection based on UAV and 3D model technology [D ] . Jinan:Shandong University , 2017 .

SHUO W , XING Z , ZHI Y , et al . Cooperative edge computing with sleep control under nonuniform traffic in mobile edge networks [J ] . IEEE Internet of Things Journal , 2019 , 6 ( 3 ): 4295 - 4306 .

SHAW J A . Radiometry and the Friis transmission equation [J ] . American Journal of Physics , 2013 , 81 ( 1 ): 33 - 37 .

DOWNEY C . Understanding wireless range calculations [J ] . Electronic Design , 2013 .

MAO Y Y , ZHANG J , SONG S H , et al . Power-delay tradeoff in multi-user mobile-edge computing systems [C ] // 2016 IEEE Global Communications Conference . Piscataway:IEEE Press , 2016 : 1 - 6 .

BURD T D , BRODERSEN R W . Processor design for portable systems [J ] . Journal of VLSI Signal Processing Systems for Signal,Image and Video Technology , 1996 , 13 ( 2/3 ): 203 - 221 .

FAN X B , WEBER W D , BARROSO L A . Power provisioning for a warehouse-sized computer [J ] . ACM SIGARCH Computer Architecture News , 2007 , 35 ( 2 ): 13 - 23 .

CHEN L X , ZHOU S , XU J . Computation peer offloading for energy-constrained mobile edge computing in small-cell networks [J ] . IEEE/ACM Transactions on Networking , 2018 , 26 ( 4 ): 1619 - 1632 .

YANG L , CAO J N , TANG S J , et al . A framework for partitioning and execution of data stream applications in mobile cloud computing [C ] // 2012 IEEE Fifth International Conference on Cloud Computing . Piscataway:IEEE Press , 2012 : 794 - 802 .

CHEN L X , XU J , ZHOU S . Computation peer offloading in mobile edge computing with energy budgets [C ] // GLOBECOM 2017 - 2017 IEEE Global Communications Conference . Piscataway:IEEE Press , 2017 : 1 - 6 .

NEELY J M . Stochastic network optimization with application to communication and queueing systems [J ] . Synthesis Lectures on Communication Networks , 2010 , 3 ( 1 ): 211 .

XU S Y , LIU Q C , GONG B , et al . RJCC:reinforcement-learningbased joint communicational-and-computational resource allocation mechanism for smart city IoT [J ] . IEEE Internet of Things Journal , 2020 , 7 ( 9 ): 8059 - 8076 .

HEESS N , DHRUVA T B , SRIRAM S , et al . Emergence of locomotion behaviours in rich environments [J ] . arXiv Preprint,arXiv:1707.02286 , 2017 .

SCHULMAN J , WOLSKI F , DHARIWAL P , et al . Proximal policy optimization algorithms [J ] . arXiv Preprint,arXiv:1707.06347 , 2017 .

王志夫 . 基于深度强化学习的双足机器人步行运动控制 [D ] . 济南:山东大学 , 2020 .

WANG Z F . Deep reinforcement learning based walking control of biped robot [D ] . Jinan:Shandong University , 2020 .

王鸿涛 . 基于强化学习的机械臂自学习控制 [D ] . 哈尔滨:哈尔滨工业大学 , 2019 .

WANG H T . Self learning control of mechanical arm based on reinforcement learning [D ] . Harbin:Harbin Institute of Technology , 2019 .

胡彦娟 . 移动边缘计算中任务卸载及资源分配算法研究 [D ] . 重庆:重庆邮电大学 , 2020 .

HU Y J . Research on task offloading and resource allocation algorithm in mobile edge computing [D ] . Chongqing:Chongqing University of Posts and Telecommunications , 2020 .

GAO Z B , WEN B , HUANG L F , et al . Q-learning-based power control for LTE enterprise femtocell networks [J ] . IEEE Systems Journal , 2017 , 11 ( 4 ): 2699 - 2707 .

LI R , ZHANG C Y , PATRAS P , et al . Learning driven mobility control of airborne base stations in emergency networks [J ] . ACM SIGMETRICS Performance Evaluation Review , 2019 , 46 ( 3 ): 163 - 166 .

KWAK J , KIM Y , LEE J , et al . DREAM:dynamic resource and task allocation for energy minimization in mobile cloud systems [J ] . IEEE Journal on Selected Areas in Communications , 2015 , 33 ( 12 ): 2510 - 2523 .

浏览量

2087

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于时间累积的低信噪比智能频谱感知方法

近端策略优化的城市环境多智能体协作对抗方法

基于大语言模型的网络威胁情报知识图谱构建技术研究

基于李雅普诺夫的RIS辅助无人机边缘计算网络中任务卸载与资源分配研究

基于近端策略优化的智能抗干扰决策算法