Split task offloading algorithm for satellite-ground integrated networks based on deep deterministic policy gradient

SONG Xiaoqin; WU Zhihao; LAI Haiguang; LEI Lei; ZHANG Lijuan; LYU Danyang; ZHENG Chenghui

doi:10.11959/j.issn.1000-436x.2024181

您当前的位置：

首页 >

文章列表页 >

Split task offloading algorithm for satellite-ground integrated networks based on deep deterministic policy gradient

Papers | 更新时间：2024-11-14

- Split task offloading algorithm for satellite-ground integrated networks based on deep deterministic policy gradient
- Journal on Communications Vol. 45, Issue 10, Pages: 116-128(2024)
- 作者机构：
  
  1.南京航空航天大学电子信息工程学院，江苏南京 210016
  2.东南大学移动通信全国重点实验室，江苏南京 210096
  3.南京控维通信科技有限公司，江苏南京 211135
- 作者简介：
- 基金信息：
  
  The National Natural Science Foundation of China(62371232);The Open Research Fund of National Mobile Communications Research Laboratory, Southeast University(2024D13);The Future Network Scientific Research Fund Project(FNSRFP-2021-ZD-4)
- DOI：10.11959/j.issn.1000-436x.2024181
  CLC： TN92
- Received：17 August 2024，
  
  Revised：2024-09-29，
  
  Published：25 October 2024
- 稿件说明：
移动端阅览
宋晓勤,吴志豪,赖海光等.基于深度确定性策略梯度的星地融合网络可拆分任务卸载算法[J].通信学报,2024,45(10):116-128.

SONG Xiaoqin,WU Zhihao,LAI Haiguang,et al.Split task offloading algorithm for satellite-ground integrated networks based on deep deterministic policy gradient[J].Journal on Communications,2024,45(10):116-128.
宋晓勤,吴志豪,赖海光等.基于深度确定性策略梯度的星地融合网络可拆分任务卸载算法[J].通信学报,2024,45(10):116-128. DOI： 10.11959/j.issn.1000-436x.2024181.

SONG Xiaoqin,WU Zhihao,LAI Haiguang,et al.Split task offloading algorithm for satellite-ground integrated networks based on deep deterministic policy gradient[J].Journal on Communications,2024,45(10):116-128. DOI： 10.11959/j.issn.1000-436x.2024181.

摘要

为解决低轨卫星网络中星地链路任务卸载时延长的问题，提出了一种基于深度确定性策略梯度（DDPG）的星地融合网络可拆分任务卸载算法。针对不同地区用户建立了星地融合网络的多接入边缘计算结构模型，通过应用多智能体DDPG算法，将系统总服务时延最小化的目标转化为智能体奖励收益最大化。在满足子任务卸载约束、服务时延约束等任务卸载约束条件下，优化用户任务拆分比例。仿真结果表明，所提算法在用户服务时延和受益用户数量等方面优于基线算法。

Abstract

To address the prolonged task offloading delay in low earth orbit satellite networks

a split task offloading algorithm based on deep deterministic policy gradient (DDPG) was proposed for satellite-ground integrated networks. A multi-access edge computing structural model of the satellite-ground integrated network was established for users in different regions. By applying a multi-agent DDPG algorithm

the objective of minimizing total system service delay was transformed into maximizing agent reward returns. Under the constraints of sub-task offloading

service delay

and other task offloading conditions

the user task splitting ratio was optimized. Simulation results indicate that the proposed algorithm outperforms baseline algorithms in terms of user service delay and the number of benefited users.

关键词

Keywords

references

杨力 , 潘成胜 , 孔相广 , 等 . 5G融合卫星网络研究综述 [J ] . 通信学报 , 2022 , 43 ( 4 ): 202 - 215 .

YANG L , PAN C S , KONG X G , et al . Review on 5G-satellite integrated network [J ] . Journal on Communications , 2022 , 43 ( 4 ): 202 - 215 .

RANAWEERA P , JURCUT A , LIYANAGE M . MEC-enabled 5G use cases: a survey on security vulnerabilities and countermeasures [J ] . ACM Computing Surveys , 2022 , 54 ( 9 ): 1 - 37 .

SPINELLI F , MANCUSO V . Toward enabled industrial verticals in 5G: a survey on MEC-based approaches to provisioning and flexibility [J ] . IEEE Communications Surveys & Tutorials , 2021 , 23 ( 1 ): 596 - 630 .

CHIEN W C , LAI C F , HOSSAIN M S , et al . Heterogeneous space and terrestrial integrated networks for IoT: architecture and challenges [J ] . IEEE Network , 2019 , 33 ( 1 ): 15 - 21 .

MOHAMMED A S , VENKATACHALAM K , HUBÁLOVSKÝ S , et al . Smart edge computing for 5G/6G satellite IOT for reducing inter transmission delay [J ] . Mobile Networks and Applications , 2022 , 27 ( 3 ): 1050 - 1059 .

RODRIGUES T K , KATO N . Network slicing with centralized and distributed reinforcement learning for combined satellite/ground networks in a 6G environment [J ] . IEEE Wireless Communications , 2022 , 29 ( 1 ): 104 - 110 .

FU S , GAO J , ZHAO L . Collaborative multi-resource allocation in terrestrial-satellite network towards 6G [J ] . IEEE Transactions on Wireless Communications , 2021 , 20 ( 11 ): 7057 - 7071 .

LIAO H J , ZHOU Z Y , ZHAO X W , et al . Learning-based queue-aware task offloading and resource allocation for space-air-ground-integrated power IoT [J ] . IEEE Internet of Things Journal , 2021 , 8 ( 7 ): 5250 - 5263 .

SHI J F , YANG H S , CHEN X , et al . Resource allocation for integrated satellite-terrestrial networks based on RSMA [J ] . IET Communications , 2024 .

TANG S Q , PAN Z S , HU G Y , et al . Deep reinforcement learning-based resource allocation for satellite Internet of things with diverse QoS guarantee [J ] . Sensors , 2022 , 22 ( 8 ): 2979 .

CHAI F R , ZHANG Q , YAO H P , et al . Joint multi-task offloading and resource allocation for mobile edge computing systems in satellite IoT [J ] . IEEE Transactions on Vehicular Technology , 2023 , 72 ( 6 ): 7783 - 7795 .

TANG Q Q , FEI Z S , LI B , et al . Computation offloading in LEO satellite networks with hybrid cloud and edge computing [J ] . IEEE Internet of Things Journal , 2021 , 8 ( 11 ): 9164 - 9176 .

黄韬 , 刘江 , 汪硕 , 等 . 未来网络技术与发展趋势综述 [J ] . 通信学报 , 2021 , 42 ( 1 ): 130 - 150 .

HUANG T , LIU J , WANG S , et al . Survey of the future network technology and trend [J ] . Journal on Communications , 2021 , 42 ( 1 ): 130 - 150 .

CHEN D , WANG W , JIANG T . New multicarrier modulation for satellite-ground transmission in space information networks [J ] . IEEE Network , 2020 , 34 ( 1 ): 101 - 107 .

KALLFASS I , HENNEBERGER R , SOMMER R , et al . High system gain E-band link in a wideband aircraft-to-ground data transmission [C ] // Proceedings of the 2019 IEEE International Conference on Microwaves, Antennas, Communications and Electronic Systems (COMCAS) . Piscataway : IEEE Press , 2019 : 1 - 5 .

CUI G F , LI X Y , XU L X , et al . Latency and energy optimization for MEC enhanced SAT-IoT networks [J ] . IEEE Access , 2020 , 8 : 55915 - 55926 .

时圣苗 , 刘全 . 采用分类经验回放的深度确定性策略梯度方法 [J ] . 自动化学报 , 2022 , 48 ( 7 ): 1816 - 1823 .

SHI S M , LIU Q . Deep deterministic policy gradient with classified experience replay [J ] . Acta Automatica Sinica , 2022 , 48 ( 7 ): 1816 - 1823 .

XING X Y , WANG S Y , LIU W J . An improved DDPG and its application in spacecraft fault knowledge graph [J ] . Sensors , 2023 , 23 ( 3 ): 1223 .

QIU C , YAO H P , YU F R , et al . Deep Q-learning aided networking, caching, and computing resources allocation in software-defined satellite-terrestrial networks [J ] . IEEE Transactions on Vehicular Technology , 2019 , 68 ( 6 ): 5871 - 5883 .

ZHANG R T , ZHAO B . Task offloading and resource allocation for cloud-edge collaboration in low earth orbit satellite networks [C ] // Proceedings of the 2023 IEEE 23rd International Conference on Communication Technology (ICCT) . Piscataway : IEEE Press , 2023 : 764 - 769 .

LI Y L , LIANG L , FU J L , et al . Multiagent reinforcement learning for task offloading of space/aerial-assisted edge computing [J ] . Security and Communication Networks , 2022 , 2022 : 4193365 .

AL-HOURANI A , GUVENC I . On modeling satellite-to-ground path-loss in urban environments [J ] . IEEE Communications Letters , 2021 , 25 ( 3 ): 696 - 700 .

Views

1059

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Joint power and spectrum optimization algorithm for ICV in diverse service scenarios

Multi-indicator intention-driven adaptive optimization strategy for UAV computation offloading and trajectory planning

Resource allocation strategy for satellite Internet integrating communication and remote sensing based on resource-time-varying graphs

Joint optimization of semantic communication reliability in integrated sensing, communication and computing networks based on regional perception

Resource allocation method for energy efficiency maximization in SWIPT IoT systems assisted by self-sustainable RIS

Related Author

Zhang Haibo

Hu Yuting

Hu Yanli

Xu Yongjun

Li Yongfu

Lin Peng

Huang Xinliang

Ning Zhaolong

Related Institution

School of Communications and Information Engineering, Chongqing University of Posts and Telecommunications

Gele Mountain Tourism Scenic Area Affairs Center

School of Automation, Chongqing University of Posts and Telecommunications

Key Laboratory of Intelligent Support Technology for Complex Environments, Ministry of Education, Nanjing University of Information Science and Technology

School of Electronic and Information Engineering, Nanjing University of Information Science and Technology

AI问答

⁰