基于深度增强学习和多目标优化改进的卫星资源分配算法

张沛; 刘帅军; 马治国; 王晓晖; 宋俊德

doi:10.11959/j.issn.1000-436x.2020117

您当前的位置：

首页 >

文章列表页 >

基于深度增强学习和多目标优化改进的卫星资源分配算法

学术论文 | 更新时间：2024-06-05

- 基于深度增强学习和多目标优化改进的卫星资源分配算法
- Improved satellite resource allocation algorithm based on DRL and MOP
- 通信学报 2020年41卷第6期页码：51-60
- 作者机构：
  
  1. 北京邮电大学计算机学院，北京 100876
  2. 中国信息通信研究院，北京 100191
  3. 中国科学院软件研究所，北京 100190
- 作者简介：
  
  [ "张沛（1986- ），女，河南三门峡人，北京邮电大学博士生，主要研究方向为卫星通信、深度增强学习、神经网络等" ]
  [ "刘帅军（1988- ），男，河北邢台人，博士，中国科学院软件研究所助理研究员，主要研究方向为低轨星座网络、卫星5G 融合、动态资源管理" ]
  [ "马治国（1978- ），男，北京人，中国信息通信研究院高级工程师，主要研究方向为5G通信、卫星通信等" ]
  [ "王晓晖（1972- ），男，浙江建德人，北京邮电大学讲师，主要研究方向为5G通信、卫星通信等" ]
  [ "宋俊德（1938- ），男，河北沧州人，博士，北京邮电大学教授，主要研究方向为智慧城市、5G通信、卫星通信等" ]
- 基金信息：
  
  国家重点研发计划基金资助项目(2018YFB0105105);国家科技重大专项基金资助项目(2018ZX03001016)
- DOI：10.11959/j.issn.1000-436x.2020117
  中图分类号： TN927+.2
- 网络首发：2020-06，
  
  纸质出版：2020-06-25
- 稿件说明：
移动端阅览
张沛, 刘帅军, 马治国, 等. 基于深度增强学习和多目标优化改进的卫星资源分配算法[J]. 通信学报, 2020,41(6):51-60.

Pei ZHANG, Shuaijun LIU, Zhiguo MA, et al. Improved satellite resource allocation algorithm based on DRL and MOP[J]. Journal on Communications, 2020, 41(6): 51-60.
张沛, 刘帅军, 马治国, 等. 基于深度增强学习和多目标优化改进的卫星资源分配算法[J]. 通信学报, 2020,41(6):51-60. DOI： 10.11959/j.issn.1000-436x.2020117.

Pei ZHANG, Shuaijun LIU, Zhiguo MA, et al. Improved satellite resource allocation algorithm based on DRL and MOP[J]. Journal on Communications, 2020, 41(6): 51-60. DOI： 10.11959/j.issn.1000-436x.2020117.

摘要

针对多波束卫星系统中资源分配序列决策的多目标优化（MOP）问题，为了在提升卫星系统性能的同时，提高用户业务需求的满意度，提出了一种基于深度增强学习（DRL）的DRL-MOP 算法。所提算法基于DRL和MOP 技术，对动态变化的系统环境和用户到达模型建模，以归一化处理后的频谱效率、能量效率和业务满意度指数的加权和作为优化目标，实现了系统和用户累计性能的优化。仿真对比表明，所提算法可以更好地解决面向多波束卫星系统的多目标优化问题，系统性能和用户满意度优化结果较好，且收敛快、复杂度低。

Abstract

In view of the multi-objective optimization (MOP) problem of sequential decision-making for resource allocations in multi-beam satellite systems

a deep reinforcement learning(DRL) based DRL-MOP algorithm was proposed to improve the system performance and user satisfaction degree.With considering the normalized weighted sum of spectrum efficiency

energy efficiency

and satisfaction index as the optimization goal

the dynamically changing system environments and user arrival model were built by the proposed algorithm

and the optimization of the accumulative performance in satellite systems based on DRL and MOP was realized.Simulation results show that the proposed algorithm can solve the MOP problem with rapid convergence ability and low complexity

and it is obviously superior to other algorithms in terms of system performance and user satisfaction optimization.

关键词

Keywords

references

易克初 , 李怡 , 孙晨华 , 等 . 卫星通信的近期发展与前景展望 [J ] . 通信学报 , 2015 , 36 ( 6 ): 161 - 176 .

YI K C , LI Y , SUN C H , et al . Recent development and its prospect of satellite communications [J ] . Journal on Communications , 2015 , 36 ( 6 ): 161 - 176 .

WANG C , CUI G , WANG W , et al . Joint estimation of carrier frequency and phase offset based on pilot symbols in quasi-constant envelope OFDM satellite systems [J ] . China Communications , 2017 , 14 ( 7 ): 1 - 11 .

史煜 , 张邦宁 , 郭道省 , 等 . 一种改进的多波束卫星通信系统功率分配算法 [J ] . 通信技术 , 2016 , 49 ( 10 ): 1355 - 1359 .

SHI Y , ZHANG B N , GUO D X , et al . A modified water-filling algorithm of power allocation for multi-beam satellite systems [J ] . Communications Technology , 2016 , 49 ( 10 ): 1355 - 1359 .

ARTIGA X , NUNEZ-MARTINEZ J, A , PEREZ-NEIRA A , et al . Terrestrial-satellite integration in dynamic 5G backhaul networks [C ] // The 8th Advanced Satellite Multimedia Systems Conference and the 14th Signal Processing for Space Communications Workshop . Piscataway:IEEE Press , 2016 : 1 - 6 .

阚茜 , 许小东 . 一种能量和频谱效率兼顾的多波束卫星系统功率分配策略 [J ] . 中国科学技术大学学报 , 2016 , 46 ( 2 ): 138 - 147 .

KAN X , XU X D . Power allocation based on energy and spectral efficiency in multi-beam satellite systems [J ] . Journal of University of Science and Technology of China , 2016 , 46 ( 2 ): 138 - 147 .

阚茜 . 衰落信道下波束卫星系统功率分配策略研究 [D ] . 合肥:中国科学技术大学 , 2016 .

KAN X . Power allocation of multi—beam satellite system in fading channel [D ] . Hefei:University of Science and Technology of China , 2016 .

COCCO G , DE COLA T , ANGELONE M , et al . Radio resource management optimization of flexible satellite payloads for DVB-S2 systems [J ] . IEEE Transactions on Broadcasting , 2018 , 64 ( 2 ): 266 - 280 .

ARAVANIS A I , SHANKAR M R B , ARAPOGLOU P , et al . Power allocation in multibeam satellite systems:a two-stage multi-objective optimization [J ] . IEEE Transactions on Wireless Communications , 2015 , 14 ( 6 ): 3171 - 3182 .

ZHANG P , WANG X , MA Z , et al . Joint optimization of satisfaction index and spectrum efficiency with cache restricted for resource allocation in multi-beam satellite systems [J ] . China Communications , 2019 , 16 ( 2 ): 189 - 201 .

廖晓闽 , 严少虎 , 石嘉 , 等 . 基于深度强化学习的蜂窝网资源分配算法 [J ] . 通信学报 , 2019 , 40 ( 2 ): 11 - 18 .

LIAO X M , YAN S H , SHI J , et al . Deep reinforcement learning based resource allocation algorithm in cellular networks [J ] . Journal on Communications , 2019 , 40 ( 2 ): 11 - 18 .

HAN Z , LEI T , LU Z , et al . Artificial intelligence based handoff management for dense WLANs:a deep reinforcement learning approach [J ] . IEEE Access , 2019 , 7 : 31688 - 31701 .

FAN H , ZHU L , YAO C , et al . Deep reinforcement learning for energy efficiency optimization in wireless networks [C ] // The 4th International Conference on Cloud Computing and Big Data Analysis . Piscataway:IEEE Press , 2019 : 465 - 471 .

FERREIRA P V R , PAFFENROTH R , WYGLINSKI A M , et al . Multi-objective reinforcement learning for cognitive satellite communications using deep neural network ensembles [J ] . IEEE Journal on Selected Areas in Communications , 2018 , 36 : 1030 - 1041 .

HU X , LIU S , WANG Y , et al . Deep reinforcement learning based beam hopping algorithm in multibeam satellite systems [J ] . IET Communications , 2019 , 13 ( 16 ): 2485 - 2491 .

HU X , ZHANG Y , LIAO X , et al . Dynamic beam hopping method based on multi-objective deep reinforcement learning for next generation satellite broadband systems [J ] . IEEE Transactions on Broadcasting , 2019 ,doi:10.1109/TBC.2019.2960940.

HU X , LIU S , CHEN R , et al . A deep reinforcement learning-based framework for dynamic resource allocation in multibeam satellite systems [J ] . IEEE Communications Letters , 2018 , 22 ( 8 ): 1612 - 1615 .

LIU S , HU X , WANG W . Deep reinforcement learning based dynamic channel allocation algorithm in multibeam satellite systems [J ] . IEEE Access , 2018 , 6 : 15733 - 15742 .

刘帅军 . 卫星通信系统中动态资源管理技术研究 [D ] . 北京:北京邮电大学 , 2018 .

LIU S J . The research on dynamic resource management techniques for satellite communication systems [D ] . Beijing:Beijing University of Posts and Telecommunications , 2018 .

彭伟 . 揭秘深度强化学习 [M ] . 北京 : 中国水利水电出版社 , 2018 : 266 - 291 .

PENG W . Exploring deep reinforcement learning [M ] . Beijing : China Water ＆ Power PressPress , 2018 : 266 - 291 .

ETSI . GEO-mobile radio interface specifications (Release 1):V1.3.1 [S ] .TS.101 376-5-5,(2005-02-11)[2019-12-20 ] .

浏览量

2051

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

智能网联汽车多样化服务场景下功率与频谱联合优化算法

UAV搭载STAR-RIS辅助的空地协同通信性能优化

多指标意图驱动的无人机计算卸载与轨迹规划自适应优化策略

基于资源时变图的通遥一体化星座资源分配策略

基于区域感知的通感算协同网络语义通信可靠性联合优化