Maritime mobile edge computing offloading method based on deep reinforcement learning

Xin SU; Leilei MENG; Yiqing ZHOU; Wu CELIMUGE

doi:10.11959/j.issn.1000-436x.2022197

您当前的位置：

首页 >

文章列表页 >

Maritime mobile edge computing offloading method based on deep reinforcement learning

Papers | 更新时间：2024-06-05

- Maritime mobile edge computing offloading method based on deep reinforcement learning
- Journal on Communications Vol. 43, Issue 10, Pages: 133-145(2022)
- 作者机构：
  
  1. 河海大学物联网工程学院，江苏常州 231022
  2. 中国科学院计算技术研究所处理器芯片全国重点实验室，北京 100190
  3. 中国科学院大学计算机科学与技术学院，北京 100190
  4. 日本电气通信大学信息理工学，东京 182-8585
- 作者简介：
- 基金信息：
  
  The National Key Research and Development Program of China(2021YFE0105500)
- DOI：10.11959/j.issn.1000-436x.2022197
  CLC： TN929.52
- Online First：2022-10，
  
  Published：25 October 2022
- 稿件说明：
移动端阅览
Xin SU, Leilei MENG, Yiqing ZHOU, et al. Maritime mobile edge computing offloading method based on deep reinforcement learning[J]. Journal on Communications, 2022, 43(10): 133-145.
DOI：

Xin SU, Leilei MENG, Yiqing ZHOU, et al. Maritime mobile edge computing offloading method based on deep reinforcement learning[J]. Journal on Communications, 2022, 43(10): 133-145. DOI： 10.11959/j.issn.1000-436x.2022197.

摘要

摘要：海洋信息系统网络节点之间的强异构特性为海洋移动边缘计算任务卸载优化带来了复杂高维度的限制条件，同时复杂多样化的海事应用会导致海洋网络局部区域出现计算任务的超负荷处理。为实现海洋网络节点计算任务的最佳卸载与资源优化，满足网络低时延、高可靠的应用服务需求，提出基于多尺度异构特征属性的海洋网络节点分层归类方法和基于深度强化学习的海洋移动边缘计算卸载方法。仿真结果表明，所提方法较传统方法能够在海洋信息系统下有效地降低网络节点的计算任务卸载时延，并且能够在大规模任务流下保持海洋网络的稳健性。

Abstract

The strong heterogeneity among the network nodes of the maritime information system brings complex and high-dimensional constraints for optimizing task offloading of the maritime mobile edge computing.The complex and diverse maritime applications also lead to the overload processing of computing tasks in local areas of the maritime network.In order to optimize the task offloading and resource management of maritime network

as well as meet the maritime application service requirements of low-latency and high-reliability

a hierarchical classification method of maritime nodes based on multi-layers attributes and a novel offloading method for maritime mobile edge computing based on deep reinforcement learning were proposed.Compared with conventional methods

simulation results show that the proposed method can effectively reduce the computing task offloading delay of the marine information system

and maintain the robustness of the maritime network with large-scale task flows.

关键词

Keywords

references

瞿逢重 , 来杭亮 , 刘建章 , 等 . 海洋物联网关键技术研究与应用 [J ] . 电信科学 , 2021 , 37 ( 7 ): 25 - 33 .

QU F Z , LAI H L , LIU J Z , et al . Research and application on key techniques of marine IoT [J ] . Telecommunications Science , 2021 , 37 ( 7 ): 25 - 33 .

YANG T , FENG H L , YANG C M , et al . Multivessel computation offloading in maritime mobile edge computing network [J ] . IEEE Internet of Things Journal , 2019 , 6 ( 3 ): 4063 - 4073 .

CHEN S Y , LUO S Y , WANGQ , et al . An intelligent task offloading algorithm (iTOA) for UAV edge computing network [J ] . Digital Communications and Networks , 2021 , 6 ( 4 ): 433 - 443 .

LIANG M Z , SU X , LIU X F , et al . Intelligent ocean convergence platform based on IoT empowered with edge computing [J ] . Journal of Internet Technology , 2020 , 21 ( 1 ): 235 - 244 .

ZHOU Y Q , LIU H , PAN Z G , et al . Two-stage cooperative multicast transmission with optimized power consumption and guaranteed coverage [J ] . IEEE Journal on Selected Areas in Communications , 2014 , 32 ( 2 ): 274 - 284 .

SHU W N , LI Y . Joint offloading strategy based on quantum particle swarm optimization for MEC-enabled vehicular networks [J ] . Digital Communications and Networks , 2022 :doi.org/10.1016/j.dcan.2022.03.009.

LIU L , ZHOU Y Q , ZHUANG W H , et al . Tractable coverage analysis for hexagonal macrocell-based heterogeneous UDNs with adaptive interference-aware CoMP [J ] . IEEE Transactions on Wireless Communications , 2019 , 18 ( 1 ): 503 - 517 .

ANBALAGAN S , KUMAR D , RAJA G , et al . SDN assisted Stackelberg game model for LTE-WiFi offloading in 5G networks [J ] . Digital Communications and Networks , 2019 , 5 ( 4 ): 268 - 275 .

EL-HABER E , NGUYEN T M , ASSI C . Joint optimization of computational cost and devices energy for task offloading in multi-tier edge-clouds [J ] . IEEE Transactions on Communications , 2019 , 67 ( 5 ): 3407 - 3421 .

NING Z L , DONG P R , KONG X J , et al . A cooperative partial computation offloading scheme for mobile edge computing enabled Internet of things [J ] . IEEE Internet of Things Journal , 2019 , 6 ( 3 ): 4804 - 4814 .

JIANG F , WEI F M , WANG J X , et al . Delay-aware energy minimization offloading scheme for mobile edge computing [C ] // Proceedings of 2020 IEEE/CIC International Conference on Communications in China (ICCC) . Piscataway:IEEE Press , 2020 : 717 - 722 .

HU X Y , WANG L F , WONG K K , et al . Edge and central cloud computing:a perfect pairing for high energy efficiency and low-latency [J ] . IEEE Transactions on Wireless Communications , 2020 , 19 ( 2 ): 1070 - 1083 .

ZHANG K , MAO Y M , LENG S P , et al . Mobile-edge computing for vehicular networks:a promising network paradigm with predictive off-loading [J ] . IEEE Vehicular Technology Magazine , 2017 , 12 ( 2 ): 36 - 44 .

DAI Y Y , XU D , MAHARJAN S , et al . Joint load balancing and offloading in vehicular edge computing and networks [J ] . IEEE Internet of Things Journal , 2019 , 6 ( 3 ): 4377 - 4387 .

HUY H V , HO T M , LE L B . Mobility-aware computation offloading in MEC-based vehicular wireless networks [J ] . IEEE Communications Letters , 2020 , 24 ( 2 ): 466 - 469 .

DU J B , YU F R , CHU X L , et al . Computation offloading and resource allocation in vehicular networks based on dual-side cost minimization [J ] . IEEE Transactions on Vehicular Technology , 2019 , 68 ( 2 ): 1079 - 1092 .

GU B , ZHOU Z Y . Task offloading in vehicular mobile edge computing:a matching-theoretic framework [J ] . IEEE Vehicular Technology Magazine , 2019 , 14 ( 3 ): 100 - 106 .

MA X T , ZHAO J H , LI Q P , et al . Reinforcement learning based task offloading and take-back in vehicle platoon networks [C ] // Proceedings of 2019 IEEE International Conference on Communications Workshops . Piscataway:IEEE Press , 2019 : 1 - 6 .

XU S Y , LIU Q C , GONG B , et al . RJCC:reinforcement-learningbased joint communicational-and-computational resource allocation mechanism for smart city IoT [J ] . IEEE Internet of Things Journal , 2020 , 7 ( 9 ): 8059 - 8076 .

KIRAN N , PAN C Y , WANG S H , et al . Joint resource allocation and computation offloading in mobile edge computing for SDN based wireless networks [J ] . Journal of Communications and Networks , 2020 , 22 ( 1 ): 1 - 11 .

JIANG F B , WANG K Z , DONG L , et al . Deep-learning-based joint resource scheduling algorithms for hybrid MEC networks [J ] . IEEE Internet of Things Journal , 2020 , 7 ( 7 ): 6252 - 6265 .

WANG R Y , JIANG X , ZHOU Y J , et al . Multi-agent reinforcement learning for edge information sharing in vehicular networks [J ] . Digital Communications and Networks , 2022 , 8 ( 3 ): 267 - 277 .

LI Y , WANG T S , WU Y , et al . Optimal dynamic spectrum allocation-assisted latency minimization for multiuser mobile edge computing [J ] . Digital Communications and Networks , 2022 , 8 ( 3 ): 247 - 256 .

YANG T T , FENG H L , GAO S , et al . Two-stage offloading optimization for energy-latency tradeoff with mobile edge computing in maritime Internet of things [J ] . IEEE Internet of Things Journal , 2020 , 7 ( 7 ): 5954 - 5963 .

苏新 , 薛淏阳 , 周一青 , 等 . 面向海洋观监测传感网的计算卸载方法研究 [J ] . 通信学报 , 2021 , 42 ( 5 ): 149 - 163 .

SU X , XUE H Y , ZHOU Y Q , et al . Research on computing offloading method for maritime observation monitoring sensor network [J ] . Journal on Communications , 2021 , 42 ( 5 ): 149 - 163 .

苏新 , 王子怡 , 王宇鹏 , 等 . 海洋观监测传感器网络多接入边缘计算卸载方法 [J ] . 物联网学报 , 2021 , 5 ( 1 ): 36 - 52 .

SU X , WANG Z Y , WANG Y P , et al . Multi-access edge computing offloading in maritime monitoring sensor networks [J ] . Chinese Journal on Internet of Things , 2021 , 5 ( 1 ): 36 - 52 .

TRAN T X , POMPILI D . Joint task offloading and resource allocation for multi-server mobile-edge computing networks [J ] . IEEE Transactions on Vehicular Technology , 2019 , 68 ( 1 ): 856 - 868 .

MENG A Q , WEI G D , ZHAO Y , et al . Green resource allocation for mobile edge computing [J ] . Digital Communications and Networks , 2022 :doi.org/10.1016/j.dcan.2022.03.001.

LIU L , ZHOU Y Q , GARCIA V , et al . Load aware joint CoMP clustering and inter-cell resource scheduling in heterogeneous ultra dense cellular networks [J ] . IEEE Transactions on Vehicular Technology , 2018 , 67 ( 3 ): 2741 - 2755 .

MNIH V , KAVUKCUOGLU K , SILVER D , et al . Human-level control through deep reinforcement learning [J ] . Nature , 2015 , 518 ( 7540 ): 529 - 533 .

WANG H D , ZHOU J , LIU W , et al . BGD-based Adam algorithm for time-domain equalizer in PAM-based optical interconnects [J ] . Optics Letters , 2020 , 45 ( 1 ): 141 .

张鹏 , 田辉 , 赵鹏涛 , 等 . 多智能体协作场景下基于强化学习值分解的计算卸载策略 [J ] . 通信学报 , 2021 , 42 ( 6 ): 1 - 15 .

ZHANG P , TIAN H , ZHAO P , et al . Computation offloading strategy in multi-agent cooperation scenario based on reinforcement learning with value-decomposition [J ] . Journal on Communications , 2021 , 42 ( 6 ): 1 - 15 .

ZHANG W W , WEN Y G , WU D O . Collaborative task execution in mobile cloud computing under a stochastic wireless channel [J ] . IEEE Transactions on Wireless Communications , 2015 , 14 ( 1 ): 81 - 93 .

Views

1571

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Joint optimization of edge computing and caching in NDN

GAT-based decision mechanism for decentralized joint routing and spectrum access

Multi-cluster computing power resource scheduling algorithm based on DDPG reinforcement learning

Task offloading and resource allocation strategy for vehicular edge computing assisted by intelligent reflecting surfaces

D3QN-based collaborative offloading algorithm for vehicular networks assisted by digital twins

Related Author

Yu ZHANG

Min CHENG

Zhou Zibo

Ren Baoquan

Zhong Xudong

Liu Qi

Qin Zhen

HU Yahui

Related Institution

School of Information and Electronics, Beijing Institute of Technology

Shanghai Institute of Mechanical and Electrical Engineering

Systems Engineering Institute, Academy of Military Science

Air Force Early Warning Academy

School of Artificial Intelligence, China University of Mining and Technology-Beijing

AI问答

⁰