Research on low earth orbit constellation beam hopping resource scheduling based on multi-agent deep reinforcement learning

ZHANG Chen; XU Yangwei; LI Wanjing; WANG Wei; ZHANG Gengxin

doi:10.11959/j.issn.1000-436x.2025009

您当前的位置：

首页 >

文章列表页 >

Research on low earth orbit constellation beam hopping resource scheduling based on multi-agent deep reinforcement learning

Papers | 更新时间：2025-02-13

- Research on low earth orbit constellation beam hopping resource scheduling based on multi-agent deep reinforcement learning
- Journal on Communications Vol. 46, Issue 1, Pages: 35-51(2025)
- 作者机构：
  
  1.南京邮电大学通信与信息工程学院，江苏南京 210003
  2.南京邮电大学通信与网络技术国家工程研究中心，江苏南京 210003
- 作者简介：
- 基金信息：
  
  The National Key Research and Development Program of China(2022YFB2902600)
- DOI：10.11959/j.issn.1000-436x.2025009
  CLC： TN927
- Received：01 August 2024，
  
  Revised：2024-12-12，
  
  Published：25 January 2025
- 稿件说明：
移动端阅览
张晨,徐阳威,李宛静等.基于多智能体深度强化学习的低轨星座跳波束资源调度研究[J].通信学报,2025,46(01):35-51.

ZHANG Chen,XU Yangwei,LI Wanjing,et al.Research on low earth orbit constellation beam hopping resource scheduling based on multi-agent deep reinforcement learning[J].Journal on Communications,2025,46(01):35-51.
张晨,徐阳威,李宛静等.基于多智能体深度强化学习的低轨星座跳波束资源调度研究[J].通信学报,2025,46(01):35-51. DOI： 10.11959/j.issn.1000-436x.2025009.

ZHANG Chen,XU Yangwei,LI Wanjing,et al.Research on low earth orbit constellation beam hopping resource scheduling based on multi-agent deep reinforcement learning[J].Journal on Communications,2025,46(01):35-51. DOI： 10.11959/j.issn.1000-436x.2025009.

摘要

针对低轨星座跳波束资源调度的需求，提出一种基于多智能体深度强化学习的低轨星座跳波束资源调度方法。通过多目标优化的选星接入方式，建立卫星与服务区域之间的映射关系。在此基础上，根据业务类型和QoS需求的多样性，采用混合专家模型的方法，构建一个资源调度多智能体，用于星上资源与跳波束图案的实时决策调度。仿真结果表明，与传统方法相比，所提资源调度方法不仅能满足不同业务对时延和吞吐量的性能需求，还能有效平衡算法的复杂度，适应多样化业务的融合传输需求，应对业务流量的时空分布不均和动态变化，具有较强的泛化能力。

Abstract

A low earth orbit constellation beam hopping resource scheduling method based on multi-agent deep reinforcement learning was proposed to meet the requirements of low earth orbit constellation beam hopping resource scheduling. The mapping relationship between the satellite and the service area was established by optimizing the access of multi-target satellite selection. On this basis

according to the diversity of service types and QoS requirements

based on the concept of mixture of experts

a resource scheduling multi-agent was constructed to carry out real-time decision scheduling of on-board resources and beam hopping patterns. The simulation results show that compared with the traditional methods

the proposed resource scheduling method can not only meet the performance requirements of different services on delay and throughput

but also effectively balance the algorithm complexity. At the same time

the algorithm can adapt to the converged transmission requirements of diversified services

cope with the uneven spatiotemporal distribution and dynamic changes of traffic and have strong generalization ability.

关键词

Keywords

references

SHENG M , ZHOU D , BAI W G , et al . 6G service coverage with mega satellite constellations [J ] . China Communications , 2022 , 19 ( 1 ): 64 - 76 .

李聪 , 何雯 , 王一帆 . 关于卫星跳波束系统的几点思考 [J ] . 空间电子技术 , 2021 , 18 ( 1 ): 8 - 13 .

LI C , HE W , WANG Y F . A study of beam-hopping technology in satellite systems [J ] . Space Electronic Technology , 2021 , 18 ( 1 ): 8 - 13 .

ANZALCHI J , COUCHMAN A , GABELLINI P , et al . Beam hopping in multi-beam broadband satellite systems: system simulation and performance comparison with non-hopped systems [C ] // Proceedings of the 2010 5th Advanced Satellite Multimedia Systems Conference and the 11th Signal Processing for Space Communications Workshop . Piscataway : IEEE Press , 2010 : 248 - 255 .

ANGELETTI P , PRIM D F , RINALDO R . Beam hopping in multi-beam broadband satellite systems: system performance and payload architecture analysis [C ] // Proceedings of the 24th AIAA International Communications Satellite Systems Conference . Reston : AIAA , 2006 : 5376 .

ALEGRE R , ALAGHA N , VÁZQUEZ-CASTRO M Á . Heuristic algorithms for flexible resource allocation in beam hopping multi-beam satellite systems [C ] // Proceedings of the 29th AIAA International Communications Satellite Systems Conference (ICSSC-2011) . Reston : AIAA , 2011 : 8001 .

ZHANG T , ZHANG L X , SHI D Y . Resource allocation in beam hopping communication system [C ] // Proceedings of the 2018 IEEE/AIAA 37th Digital Avionics Systems Conference (DASC) . Piscataway : IEEE Press , 2018 : 1 - 5 .

HAN H , ZHENG X Q , HUANG Q F , et al . QoS-equilibrium slot allocation for beam hopping in broadband satellite communication systems [J ] . Wireless Networks , 2015 , 21 ( 8 ): 2617 - 2630 .

TANG J Y , BIAN D M , LI G X , et al . Optimization method of dynamic beam position for LEO beam-hopping satellite communication systems [J ] . IEEE Access , 2021 , 9 : 57578 - 57588 .

XU G L , TAN F , ZHAO Y Y , et al . Joint beam-hopping scheduling and coverage control in multibeam satellite systems [J ] . IEEE Wireless Communications Letters , 20 , 12 ( 2 ): 267 - 271 .

HU X , LIU S J , WANG Y P , et al . Deep reinforcement learning-based beam hopping algorithm in multibeam satellite systems [J ] . IET Communications , 2019 , 13 ( 16 ): 2485 - 2491 .

HAN Y F , ZHANG C , ZHANG G X . Dynamic beam hopping resource allocation algorithm based on deep reinforcement learning in multi-beam satellite systems [C ] // Proceedings of the 2021 3rd International Academic Exchange Conference on Science and Technology Innovation (IAECST) . Piscataway : IEEE Press , 2021 : 68 - 73 .

CAO Y , LIEN S Y , LIANG Y C , et al . Collaborative deep reinforcement learning for resource optimization in non-terrestrial networks [C ] // Proceedings of the 2023 IEEE 34th Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC) . Piscataway : IEEE Press , 2023 : 1 - 7 .

张沛 , 刘帅军 , 马治国 , 等 . 基于深度增强学习和多目标优化改进的卫星资源分配算法 [J ] . 通信学报 , 2020 , 41 ( 6 ): 51 - 60 .

ZHANG P , LIU S J , MA Z G , et al . Improved satellite resource allocation algorithm based on DRL and MOP [J ] . Journal on Communications , 2020 , 41 ( 6 ): 51 - 60 .

HU X , ZHANG Y C , LIAO X L , et al . Dynamic beam hopping method based on multi-objective deep reinforcement learning for next generation satellite broadband systems [J ] . IEEE Transactions on Broadcasting , 2020 , 66 ( 3 ): 630 - 646 .

LIN Z Y , NI Z Y , KUANG L L , et al . Satellite-terrestrial coordinated multi-satellite beam hopping scheduling based on multi-agent deep reinforcement learning [J ] . IEEE Transactions on Wireless Communications , 2024 , 23 ( 8 ): 10091 - 10103 .

罗铭 , 詹骐榜 , 邱敏蓉 . 面向V2I通信的异构跨域条件隐私保护环签密方案 [J ] . 信息网络安全 , 2024 , 24 ( 7 ): 993 - 1005 .

LUO M , ZHAN Q B , QIU M R . A heterogeneous cross-domain conditional privacy protection ring signcryption scheme for V2I communication [J ] . Netinfo Security , 2024 , 24 ( 7 ): 993 - 1005 .

彭明阳 . 面向低轨星座的跳波束机制及资源分配研究 [D ] . 南京 : 南京邮电大学 , 2023 .

PENG M Y . Research on beam hopping mechanism and resource allocation for LEO constellation [D ] . Nanjing : Nanjing University of Posts and Telecommunications , 2023 .

黄飞 . 低轨卫星通信接入与切换策略研究 [D ] . 成都 : 电子科技大学 , 2009 .

HUANG F . Research on access and handover strategy of LEO satellite communication [D ] . Chengdu : University of Electronic Science and Technology of China , 2009 .

张凤磊 . 多层卫星网络系统接入选择技术研究 [D ] . 西安 : 西安电子科技大学 , 2020 .

ZHANG F L . Research on access selection technology of multi-layer satellite network system [D ] . Xi’an : Xidian University , 2020 .

潘成胜 , 王羽夫 , 杨力 . 基于改进LSTM算法的天地一体化信息网络流量预测 [J ] . 天地一体化信息网络 , 2020 , 1 ( 2 ): 57 - 65 .

PAN C S , WANG Y F , YANG L . Traffic prediction of space-integrated-ground information network based on improved LSTM algorithm [J ] . Space-Integrated-Ground Information Networks , 2020 , 1 ( 2 ): 57 - 65 .

CAINI C , CORAZZA G E , FALCIASECCA G , et al . A spectrum-and power-efficient EHF mobile satellite system to be integrated with terrestrial cellular systems [J ] . IEEE Journal on Selected Areas in Communications , 2002 , 10 ( 8 ): 1315 - 1325 .

THYLWE K E , MCCABE P . On calculations of Legendre functions and associated Legendre functions of the first kind of complex degree [J ] . Communications in Theoretical Physics , 2015 , 64 ( 7 ): 9 - 12 .

HERZ C S . Bessel functions of matrix argument [J ] . The Annals of Mathematics , 1955 , 61 ( 3 ): 474 .

张晨 , 彭明阳 , 张更新 . 基于联合优化的高通量卫星跳波束图案设计研究 [J ] . 南京邮电大学学报(自然科学版) , 2021 , 41 ( 3 ): 1 - 8 .

ZHANG C , PENG M Y , ZHANG G X . Beam hopping pattern method for high-throughput satellite based on joint optimization [J ] . Journal of Nanjing University of Posts and Telecommunications (Natural Science Edition) , 2021 , 41 ( 3 ): 1 - 8 .

VIDAL F , LEGAY H , GOUSSETIS G , et al . A methodology to benchmark flexible payload architectures in a megaconstellation use case [J ] . International Journal of Satellite Communications and Networking , 2021 , 39 ( 1 ): 29 - 46 .

3GPP TS 38.214(v18.4.0). NR physical layer procedures for data [S ] . 2024 .

3GPP R1-070674. LTE physical layer framework for performance verification [S ] . 2007 .

张雨晨 . 基于多目标深度强化学习的多波束卫星动态波束调度算法研究 [D ] . 北京 : 北京邮电大学 , 2020 .

ZHANG Y C . Research on dynamic beam scheduling algorithm of multi-beam satellite based on multi-objective depth reinforcement learning [D ] . Beijing : Beijing University of Posts and Telecommunications , 2020 .

ZADEH L A . Optimality and non-scalar-valued performance criteria [J ] . IEEE Transactions on Automatic Control , 1963 , 8 ( 1 ): 59 - 60 .

TAJMAJER T . Modular multi-objective deep reinforcement learning with decision values [C ] // Proceedings of the 2018 Federated Conference on Computer Science and Information Systems . Piscataway : IEEE Press , 2018 : 85 - 93 .

KAELBLING L P , LITTMAN M L , MOORE A W . Reinforcement learning: a survey [J ] . Journal of Artificial Intelligence Research , 1996 , 4 ( 1 ): 237 - 285 .

韩永锋 . GEO卫星通信系统跳波束资源动态分配方法研究 [D ] . 南京 : 南京邮电大学 , 2022 .

HAN Y F . Research on dynamic allocation method of beam hopping resources in GEO satellite communication system [D ] . Nanjing : Nanjing University of Posts and Telecommunications , 2022 .

董豪 , 丁子涵 , 仉尚航 , 等 . 深度强化学习: 基础、研究与应用 [M ] . 北京 : 电子工业出版社 , 2021 .

DONG H , DING Z H , ZHANG S H , et al . Deep reinforcement learning: fundamentals, research and applications [M ] . Beijing : Publishing House of Electronics Industry , 2021 .

Views

1590

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Multi-cluster computing power resource scheduling algorithm based on DDPG reinforcement learning

Storage resource scheduling optimization method for separated data center based on deep reinforcement learning

GAT-based decision mechanism for decentralized joint routing and spectrum access

Related Author

HU Yahui

WANG Yuelin

ZHANG Chenkang

HONG Yuchen

FAN Pengfei

SONG Junping

ZHOU Xu

BAO Zhen

Related Institution

School of Artificial Intelligence, China University of Mining and Technology-Beijing

Computer Network information Center, Chinese Academy of Sciences

CHN Energy Zhi Shen Control Technology Co., Ltd.

State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications

China Electric Power Research Institute

AI问答

⁰