基于深度强化学习的海洋移动边缘计算卸载方法

苏新; 孟蕾蕾; 周一青; CELIMUGE Wu

doi:10.11959/j.issn.1000-436x.2022197

您当前的位置：

首页 >

文章列表页 >

基于深度强化学习的海洋移动边缘计算卸载方法

学术论文 | 更新时间：2024-06-05

- 基于深度强化学习的海洋移动边缘计算卸载方法
- Maritime mobile edge computing offloading method based on deep reinforcement learning
- 通信学报 2022年43卷第10期页码：133-145
- 作者机构：
  
  1. 河海大学物联网工程学院，江苏常州 231022
  2. 中国科学院计算技术研究所处理器芯片全国重点实验室，北京 100190
  3. 中国科学院大学计算机科学与技术学院，北京 100190
  4. 日本电气通信大学信息理工学，东京 182-8585
- 作者简介：
  
  [ "苏新（1986− ），男，河北霸州人，博士，河海大学教授，主要研究方向为移动通信、边缘/雾计算、智慧海洋等" ]
  [ "孟蕾蕾（1996− ），女，甘肃定西人，河海大学硕士生，主要研究方向为移动通信、边缘/雾计算、智慧海洋等" ]
  [ "周一青（1975− ），女，浙江杭州人，中国科学院计算技术研究所研究员，主要研究方向为通信与计算融合、移动边缘计算、存储通信、干扰管控等" ]
  [ "CELIMUGE Wu（1979− ），男，博士，日本电气通信大学教授，主要研究方向为无线网络、物联网系统、边缘计算等" ]
- 基金信息：
  
  国家重点研发计划基金资助项目(2021YFE0105500)
- DOI：10.11959/j.issn.1000-436x.2022197
  中图分类号： TN929.52
- 网络首发：2022-10，
  
  纸质出版：2022-10-25
- 稿件说明：
移动端阅览
苏新, 孟蕾蕾, 周一青, 等. 基于深度强化学习的海洋移动边缘计算卸载方法[J]. 通信学报, 2022,43(10):133-145.

Xin SU, Leilei MENG, Yiqing ZHOU, et al. Maritime mobile edge computing offloading method based on deep reinforcement learning[J]. Journal on Communications, 2022, 43(10): 133-145.
苏新, 孟蕾蕾, 周一青, 等. 基于深度强化学习的海洋移动边缘计算卸载方法[J]. 通信学报, 2022,43(10):133-145. DOI： 10.11959/j.issn.1000-436x.2022197.

Xin SU, Leilei MENG, Yiqing ZHOU, et al. Maritime mobile edge computing offloading method based on deep reinforcement learning[J]. Journal on Communications, 2022, 43(10): 133-145. DOI： 10.11959/j.issn.1000-436x.2022197.

摘要

摘要：海洋信息系统网络节点之间的强异构特性为海洋移动边缘计算任务卸载优化带来了复杂高维度的限制条件，同时复杂多样化的海事应用会导致海洋网络局部区域出现计算任务的超负荷处理。为实现海洋网络节点计算任务的最佳卸载与资源优化，满足网络低时延、高可靠的应用服务需求，提出基于多尺度异构特征属性的海洋网络节点分层归类方法和基于深度强化学习的海洋移动边缘计算卸载方法。仿真结果表明，所提方法较传统方法能够在海洋信息系统下有效地降低网络节点的计算任务卸载时延，并且能够在大规模任务流下保持海洋网络的稳健性。

Abstract

The strong heterogeneity among the network nodes of the maritime information system brings complex and high-dimensional constraints for optimizing task offloading of the maritime mobile edge computing.The complex and diverse maritime applications also lead to the overload processing of computing tasks in local areas of the maritime network.In order to optimize the task offloading and resource management of maritime network

as well as meet the maritime application service requirements of low-latency and high-reliability

a hierarchical classification method of maritime nodes based on multi-layers attributes and a novel offloading method for maritime mobile edge computing based on deep reinforcement learning were proposed.Compared with conventional methods

simulation results show that the proposed method can effectively reduce the computing task offloading delay of the marine information system

and maintain the robustness of the maritime network with large-scale task flows.

关键词

Keywords

references

瞿逢重 , 来杭亮 , 刘建章 , 等 . 海洋物联网关键技术研究与应用 [J ] . 电信科学 , 2021 , 37 ( 7 ): 25 - 33 .

QU F Z , LAI H L , LIU J Z , et al . Research and application on key techniques of marine IoT [J ] . Telecommunications Science , 2021 , 37 ( 7 ): 25 - 33 .

YANG T , FENG H L , YANG C M , et al . Multivessel computation offloading in maritime mobile edge computing network [J ] . IEEE Internet of Things Journal , 2019 , 6 ( 3 ): 4063 - 4073 .

CHEN S Y , LUO S Y , WANGQ , et al . An intelligent task offloading algorithm (iTOA) for UAV edge computing network [J ] . Digital Communications and Networks , 2021 , 6 ( 4 ): 433 - 443 .

LIANG M Z , SU X , LIU X F , et al . Intelligent ocean convergence platform based on IoT empowered with edge computing [J ] . Journal of Internet Technology , 2020 , 21 ( 1 ): 235 - 244 .

ZHOU Y Q , LIU H , PAN Z G , et al . Two-stage cooperative multicast transmission with optimized power consumption and guaranteed coverage [J ] . IEEE Journal on Selected Areas in Communications , 2014 , 32 ( 2 ): 274 - 284 .

SHU W N , LI Y . Joint offloading strategy based on quantum particle swarm optimization for MEC-enabled vehicular networks [J ] . Digital Communications and Networks , 2022 :doi.org/10.1016/j.dcan.2022.03.009.

LIU L , ZHOU Y Q , ZHUANG W H , et al . Tractable coverage analysis for hexagonal macrocell-based heterogeneous UDNs with adaptive interference-aware CoMP [J ] . IEEE Transactions on Wireless Communications , 2019 , 18 ( 1 ): 503 - 517 .

ANBALAGAN S , KUMAR D , RAJA G , et al . SDN assisted Stackelberg game model for LTE-WiFi offloading in 5G networks [J ] . Digital Communications and Networks , 2019 , 5 ( 4 ): 268 - 275 .

EL-HABER E , NGUYEN T M , ASSI C . Joint optimization of computational cost and devices energy for task offloading in multi-tier edge-clouds [J ] . IEEE Transactions on Communications , 2019 , 67 ( 5 ): 3407 - 3421 .

NING Z L , DONG P R , KONG X J , et al . A cooperative partial computation offloading scheme for mobile edge computing enabled Internet of things [J ] . IEEE Internet of Things Journal , 2019 , 6 ( 3 ): 4804 - 4814 .

JIANG F , WEI F M , WANG J X , et al . Delay-aware energy minimization offloading scheme for mobile edge computing [C ] // Proceedings of 2020 IEEE/CIC International Conference on Communications in China (ICCC) . Piscataway:IEEE Press , 2020 : 717 - 722 .

HU X Y , WANG L F , WONG K K , et al . Edge and central cloud computing:a perfect pairing for high energy efficiency and low-latency [J ] . IEEE Transactions on Wireless Communications , 2020 , 19 ( 2 ): 1070 - 1083 .

ZHANG K , MAO Y M , LENG S P , et al . Mobile-edge computing for vehicular networks:a promising network paradigm with predictive off-loading [J ] . IEEE Vehicular Technology Magazine , 2017 , 12 ( 2 ): 36 - 44 .

DAI Y Y , XU D , MAHARJAN S , et al . Joint load balancing and offloading in vehicular edge computing and networks [J ] . IEEE Internet of Things Journal , 2019 , 6 ( 3 ): 4377 - 4387 .

HUY H V , HO T M , LE L B . Mobility-aware computation offloading in MEC-based vehicular wireless networks [J ] . IEEE Communications Letters , 2020 , 24 ( 2 ): 466 - 469 .

DU J B , YU F R , CHU X L , et al . Computation offloading and resource allocation in vehicular networks based on dual-side cost minimization [J ] . IEEE Transactions on Vehicular Technology , 2019 , 68 ( 2 ): 1079 - 1092 .

GU B , ZHOU Z Y . Task offloading in vehicular mobile edge computing:a matching-theoretic framework [J ] . IEEE Vehicular Technology Magazine , 2019 , 14 ( 3 ): 100 - 106 .

MA X T , ZHAO J H , LI Q P , et al . Reinforcement learning based task offloading and take-back in vehicle platoon networks [C ] // Proceedings of 2019 IEEE International Conference on Communications Workshops . Piscataway:IEEE Press , 2019 : 1 - 6 .

XU S Y , LIU Q C , GONG B , et al . RJCC:reinforcement-learningbased joint communicational-and-computational resource allocation mechanism for smart city IoT [J ] . IEEE Internet of Things Journal , 2020 , 7 ( 9 ): 8059 - 8076 .

KIRAN N , PAN C Y , WANG S H , et al . Joint resource allocation and computation offloading in mobile edge computing for SDN based wireless networks [J ] . Journal of Communications and Networks , 2020 , 22 ( 1 ): 1 - 11 .

JIANG F B , WANG K Z , DONG L , et al . Deep-learning-based joint resource scheduling algorithms for hybrid MEC networks [J ] . IEEE Internet of Things Journal , 2020 , 7 ( 7 ): 6252 - 6265 .

WANG R Y , JIANG X , ZHOU Y J , et al . Multi-agent reinforcement learning for edge information sharing in vehicular networks [J ] . Digital Communications and Networks , 2022 , 8 ( 3 ): 267 - 277 .

LI Y , WANG T S , WU Y , et al . Optimal dynamic spectrum allocation-assisted latency minimization for multiuser mobile edge computing [J ] . Digital Communications and Networks , 2022 , 8 ( 3 ): 247 - 256 .

YANG T T , FENG H L , GAO S , et al . Two-stage offloading optimization for energy-latency tradeoff with mobile edge computing in maritime Internet of things [J ] . IEEE Internet of Things Journal , 2020 , 7 ( 7 ): 5954 - 5963 .

苏新 , 薛淏阳 , 周一青 , 等 . 面向海洋观监测传感网的计算卸载方法研究 [J ] . 通信学报 , 2021 , 42 ( 5 ): 149 - 163 .

SU X , XUE H Y , ZHOU Y Q , et al . Research on computing offloading method for maritime observation monitoring sensor network [J ] . Journal on Communications , 2021 , 42 ( 5 ): 149 - 163 .

苏新 , 王子怡 , 王宇鹏 , 等 . 海洋观监测传感器网络多接入边缘计算卸载方法 [J ] . 物联网学报 , 2021 , 5 ( 1 ): 36 - 52 .

SU X , WANG Z Y , WANG Y P , et al . Multi-access edge computing offloading in maritime monitoring sensor networks [J ] . Chinese Journal on Internet of Things , 2021 , 5 ( 1 ): 36 - 52 .

TRAN T X , POMPILI D . Joint task offloading and resource allocation for multi-server mobile-edge computing networks [J ] . IEEE Transactions on Vehicular Technology , 2019 , 68 ( 1 ): 856 - 868 .

MENG A Q , WEI G D , ZHAO Y , et al . Green resource allocation for mobile edge computing [J ] . Digital Communications and Networks , 2022 :doi.org/10.1016/j.dcan.2022.03.001.

LIU L , ZHOU Y Q , GARCIA V , et al . Load aware joint CoMP clustering and inter-cell resource scheduling in heterogeneous ultra dense cellular networks [J ] . IEEE Transactions on Vehicular Technology , 2018 , 67 ( 3 ): 2741 - 2755 .

MNIH V , KAVUKCUOGLU K , SILVER D , et al . Human-level control through deep reinforcement learning [J ] . Nature , 2015 , 518 ( 7540 ): 529 - 533 .

WANG H D , ZHOU J , LIU W , et al . BGD-based Adam algorithm for time-domain equalizer in PAM-based optical interconnects [J ] . Optics Letters , 2020 , 45 ( 1 ): 141 .

张鹏 , 田辉 , 赵鹏涛 , 等 . 多智能体协作场景下基于强化学习值分解的计算卸载策略 [J ] . 通信学报 , 2021 , 42 ( 6 ): 1 - 15 .

ZHANG P , TIAN H , ZHAO P , et al . Computation offloading strategy in multi-agent cooperation scenario based on reinforcement learning with value-decomposition [J ] . Journal on Communications , 2021 , 42 ( 6 ): 1 - 15 .

ZHANG W W , WEN Y G , WU D O . Collaborative task execution in mobile cloud computing under a stochastic wireless channel [J ] . IEEE Transactions on Wireless Communications , 2015 , 14 ( 1 ): 81 - 93 .

浏览量

1487

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

NDN中边缘计算与缓存的联合优化

基于DDPG强化学习的多集群算力资源调度算法

智能反射表面辅助的车载边缘任务卸载和资源分配策略

数字孪生辅助下基于D3QN的车载网络协同卸载算法

基于深度强化学习的分离式数据中心存储资源调度优化方法