基于深度确定性策略梯度的星地融合网络可拆分任务卸载算法

宋晓勤; 吴志豪; 赖海光; 雷磊; 张莉涓; 吕丹阳; 郑成辉

doi:10.11959/j.issn.1000-436x.2024181

您当前的位置：

首页 >

文章列表页 >

基于深度确定性策略梯度的星地融合网络可拆分任务卸载算法

学术论文 | 更新时间：2024-11-14

- 基于深度确定性策略梯度的星地融合网络可拆分任务卸载算法
- Split task offloading algorithm for satellite-ground integrated networks based on deep deterministic policy gradient
- 通信学报 2024年45卷第10期页码：116-128
- 作者机构：
  
  1.南京航空航天大学电子信息工程学院，江苏南京 210016
  2.东南大学移动通信全国重点实验室，江苏南京 210096
  3.南京控维通信科技有限公司，江苏南京 211135
- 作者简介：
  
  [ "宋晓勤（1973- ），女，江苏扬州人，博士，南京航空航天大学副教授、硕士生导师，主要研究方向为智能组网与协同技术等。" ]
  [ "吴志豪（2000- ），男，湖南邵阳人，南京航空航天大学硕士生，主要研究方向为多接入边缘计算、多目标资源联合分配等。" ]
  [ "赖海光（1975- ），男，江苏南京人，南京控维通信科技有限公司高级工程师，主要研究方向为卫星通信技术等。" ]
  [ "雷磊（1981- ），男，江西南昌人，博士，南京航空航天大学教授、博士生导师，主要研究方向为航空平台组网技术、智能无人机集群技术等。" ]
  [ "张莉涓（1988- ），女，四川广安人，博士，南京航空航天大学副教授、硕士生导师，主要研究方向为无线泛在网络接入技术、智能无人机集群技术等。" ]
  [ "吕丹阳（1999- ），男，山东青岛人，南京航空航天大学硕士生，主要研究方向为多目标强化学习训练、多目标资源联合分配等。" ]
  [ "郑成辉（1977- ），男，江苏南京人，南京控维通信科技有限公司工程师，主要研究方向为卫星通信技术等。" ]
- 基金信息：
  
  国家自然科学基金资助项目(62371232);东南大学移动通信全国重点实验室开放研究基金资助项目(2024D13);未来网络科研基金资助项目(FNSRFP-2021-ZD-4)
- DOI：10.11959/j.issn.1000-436x.2024181
  中图分类号： TN92
- 收稿：2024-08-17，
  
  修回：2024-09-29，
  
  纸质出版：2024-10-25
- 稿件说明：
移动端阅览
宋晓勤,吴志豪,赖海光等.基于深度确定性策略梯度的星地融合网络可拆分任务卸载算法[J].通信学报,2024,45(10):116-128.

SONG Xiaoqin,WU Zhihao,LAI Haiguang,et al.Split task offloading algorithm for satellite-ground integrated networks based on deep deterministic policy gradient[J].Journal on Communications,2024,45(10):116-128.
宋晓勤,吴志豪,赖海光等.基于深度确定性策略梯度的星地融合网络可拆分任务卸载算法[J].通信学报,2024,45(10):116-128. DOI： 10.11959/j.issn.1000-436x.2024181.

SONG Xiaoqin,WU Zhihao,LAI Haiguang,et al.Split task offloading algorithm for satellite-ground integrated networks based on deep deterministic policy gradient[J].Journal on Communications,2024,45(10):116-128. DOI： 10.11959/j.issn.1000-436x.2024181.

摘要

为解决低轨卫星网络中星地链路任务卸载时延长的问题，提出了一种基于深度确定性策略梯度（DDPG）的星地融合网络可拆分任务卸载算法。针对不同地区用户建立了星地融合网络的多接入边缘计算结构模型，通过应用多智能体DDPG算法，将系统总服务时延最小化的目标转化为智能体奖励收益最大化。在满足子任务卸载约束、服务时延约束等任务卸载约束条件下，优化用户任务拆分比例。仿真结果表明，所提算法在用户服务时延和受益用户数量等方面优于基线算法。

Abstract

To address the prolonged task offloading delay in low earth orbit satellite networks

a split task offloading algorithm based on deep deterministic policy gradient (DDPG) was proposed for satellite-ground integrated networks. A multi-access edge computing structural model of the satellite-ground integrated network was established for users in different regions. By applying a multi-agent DDPG algorithm

the objective of minimizing total system service delay was transformed into maximizing agent reward returns. Under the constraints of sub-task offloading

service delay

and other task offloading conditions

the user task splitting ratio was optimized. Simulation results indicate that the proposed algorithm outperforms baseline algorithms in terms of user service delay and the number of benefited users.

关键词

Keywords

references

杨力 , 潘成胜 , 孔相广 , 等 . 5G融合卫星网络研究综述 [J ] . 通信学报 , 2022 , 43 ( 4 ): 202 - 215 .

YANG L , PAN C S , KONG X G , et al . Review on 5G-satellite integrated network [J ] . Journal on Communications , 2022 , 43 ( 4 ): 202 - 215 .

RANAWEERA P , JURCUT A , LIYANAGE M . MEC-enabled 5G use cases: a survey on security vulnerabilities and countermeasures [J ] . ACM Computing Surveys , 2022 , 54 ( 9 ): 1 - 37 .

SPINELLI F , MANCUSO V . Toward enabled industrial verticals in 5G: a survey on MEC-based approaches to provisioning and flexibility [J ] . IEEE Communications Surveys & Tutorials , 2021 , 23 ( 1 ): 596 - 630 .

CHIEN W C , LAI C F , HOSSAIN M S , et al . Heterogeneous space and terrestrial integrated networks for IoT: architecture and challenges [J ] . IEEE Network , 2019 , 33 ( 1 ): 15 - 21 .

MOHAMMED A S , VENKATACHALAM K , HUBÁLOVSKÝ S , et al . Smart edge computing for 5G/6G satellite IOT for reducing inter transmission delay [J ] . Mobile Networks and Applications , 2022 , 27 ( 3 ): 1050 - 1059 .

RODRIGUES T K , KATO N . Network slicing with centralized and distributed reinforcement learning for combined satellite/ground networks in a 6G environment [J ] . IEEE Wireless Communications , 2022 , 29 ( 1 ): 104 - 110 .

FU S , GAO J , ZHAO L . Collaborative multi-resource allocation in terrestrial-satellite network towards 6G [J ] . IEEE Transactions on Wireless Communications , 2021 , 20 ( 11 ): 7057 - 7071 .

LIAO H J , ZHOU Z Y , ZHAO X W , et al . Learning-based queue-aware task offloading and resource allocation for space-air-ground-integrated power IoT [J ] . IEEE Internet of Things Journal , 2021 , 8 ( 7 ): 5250 - 5263 .

SHI J F , YANG H S , CHEN X , et al . Resource allocation for integrated satellite-terrestrial networks based on RSMA [J ] . IET Communications , 2024 .

TANG S Q , PAN Z S , HU G Y , et al . Deep reinforcement learning-based resource allocation for satellite Internet of things with diverse QoS guarantee [J ] . Sensors , 2022 , 22 ( 8 ): 2979 .

CHAI F R , ZHANG Q , YAO H P , et al . Joint multi-task offloading and resource allocation for mobile edge computing systems in satellite IoT [J ] . IEEE Transactions on Vehicular Technology , 2023 , 72 ( 6 ): 7783 - 7795 .

TANG Q Q , FEI Z S , LI B , et al . Computation offloading in LEO satellite networks with hybrid cloud and edge computing [J ] . IEEE Internet of Things Journal , 2021 , 8 ( 11 ): 9164 - 9176 .

黄韬 , 刘江 , 汪硕 , 等 . 未来网络技术与发展趋势综述 [J ] . 通信学报 , 2021 , 42 ( 1 ): 130 - 150 .

HUANG T , LIU J , WANG S , et al . Survey of the future network technology and trend [J ] . Journal on Communications , 2021 , 42 ( 1 ): 130 - 150 .

CHEN D , WANG W , JIANG T . New multicarrier modulation for satellite-ground transmission in space information networks [J ] . IEEE Network , 2020 , 34 ( 1 ): 101 - 107 .

KALLFASS I , HENNEBERGER R , SOMMER R , et al . High system gain E-band link in a wideband aircraft-to-ground data transmission [C ] // Proceedings of the 2019 IEEE International Conference on Microwaves, Antennas, Communications and Electronic Systems (COMCAS) . Piscataway : IEEE Press , 2019 : 1 - 5 .

CUI G F , LI X Y , XU L X , et al . Latency and energy optimization for MEC enhanced SAT-IoT networks [J ] . IEEE Access , 2020 , 8 : 55915 - 55926 .

时圣苗 , 刘全 . 采用分类经验回放的深度确定性策略梯度方法 [J ] . 自动化学报 , 2022 , 48 ( 7 ): 1816 - 1823 .

SHI S M , LIU Q . Deep deterministic policy gradient with classified experience replay [J ] . Acta Automatica Sinica , 2022 , 48 ( 7 ): 1816 - 1823 .

XING X Y , WANG S Y , LIU W J . An improved DDPG and its application in spacecraft fault knowledge graph [J ] . Sensors , 2023 , 23 ( 3 ): 1223 .

QIU C , YAO H P , YU F R , et al . Deep Q-learning aided networking, caching, and computing resources allocation in software-defined satellite-terrestrial networks [J ] . IEEE Transactions on Vehicular Technology , 2019 , 68 ( 6 ): 5871 - 5883 .

ZHANG R T , ZHAO B . Task offloading and resource allocation for cloud-edge collaboration in low earth orbit satellite networks [C ] // Proceedings of the 2023 IEEE 23rd International Conference on Communication Technology (ICCT) . Piscataway : IEEE Press , 2023 : 764 - 769 .

LI Y L , LIANG L , FU J L , et al . Multiagent reinforcement learning for task offloading of space/aerial-assisted edge computing [J ] . Security and Communication Networks , 2022 , 2022 : 4193365 .

AL-HOURANI A , GUVENC I . On modeling satellite-to-ground path-loss in urban environments [J ] . IEEE Communications Letters , 2021 , 25 ( 3 ): 696 - 700 .

浏览量

1059

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

智能网联汽车多样化服务场景下功率与频谱联合优化算法

多指标意图驱动的无人机计算卸载与轨迹规划自适应优化策略

基于资源时变图的通遥一体化星座资源分配策略

基于区域感知的通感算协同网络语义通信可靠性联合优化

面向能效最大化的自供电RIS辅助SWIPT物联网资源分配方法