基于样本信息熵辅助的深度强化学习抗干扰策略

李刚; 吴麒; 王翔; 罗皓; 李良鸿; 景小荣; 陈前斌

doi:10.11959/j.issn.1000-436x.2024161

您当前的位置：

首页 >

文章列表页 >

基于样本信息熵辅助的深度强化学习抗干扰策略

学术论文 | 更新时间：2024-10-10

- 基于样本信息熵辅助的深度强化学习抗干扰策略
- Deep reinforcement learning-empowered anti-jamming strategy aided by sample information entropy
- 通信学报 2024年45卷第9期页码：115-128
- 作者机构：
  
  1.中国西南电子技术研究所，四川成都 610036
  2.重庆邮电大学通信与信息工程学院，重庆 400065
- 作者简介：
  
  [ "李刚（1989- ），男，重庆人，中国西南电子技术研究所工程师，主要研究方向为分布式通信系统设计及人工智能等。" ]
  [ "吴麒（1985- ），男，四川眉山人，博士，中国西南电子技术研究所高级工程师，主要研究方向为通信与系统总体设计、智能通信技术等。" ]
  [ "王翔（1988- ），男，湖南常德人，博士，中国西南电子技术研究所工程师，主要研究方向为智能通信与网络技术等。" ]
  [ "罗皓（1997- ），男，四川永川人，中国西南电子技术研究所工程师，主要研究方向为干扰对抗、信号处理等。" ]
  [ "李良鸿（1992- ），男，四川巴中人，重庆邮电大学博士生，主要研究方向为通信抗干扰。" ]
  [ "景小荣（1974- ），男，甘肃平凉人，博士，重庆邮电大学教授、博士生导师，主要研究方向为无线通信及通信对抗等。" ]
  [ "陈前斌（1967- ），男，四川营山人，博士，重庆邮电大学教授、博士生导师，主要研究方向为无线通信、多媒体信息传输与处理。" ]
- 基金信息：
  
  国家自然科学基金资助项目;The National Natural Science Foundation of China(U23A20279);中电天奥创新理论技术群基金资助项目(2022-1193-04-04)
- DOI：10.11959/j.issn.1000-436x.2024161
  中图分类号： TN975
- 收稿：2024-02-05，
  
  修回：2024-08-06，
  
  纸质出版：2024-09-25
- 稿件说明：
移动端阅览
李刚,吴麒,王翔等.基于样本信息熵辅助的深度强化学习抗干扰策略[J].通信学报,2024,45(09):115-128.

LI Gang,WU Qi,WANG Xiang,et al.Deep reinforcement learning-empowered anti-jamming strategy aided by sample information entropy[J].Journal on Communications,2024,45(09):115-128.
李刚,吴麒,王翔等.基于样本信息熵辅助的深度强化学习抗干扰策略[J].通信学报,2024,45(09):115-128. DOI： 10.11959/j.issn.1000-436x.2024161.

LI Gang,WU Qi,WANG Xiang,et al.Deep reinforcement learning-empowered anti-jamming strategy aided by sample information entropy[J].Journal on Communications,2024,45(09):115-128. DOI： 10.11959/j.issn.1000-436x.2024161.

摘要

针对深度强化学习驱动的智能化干扰，提出了一种基于样本信息熵辅助的通信抗干扰策略。首先，基于神经网络对抗干扰策略网络和熵预测网络进行设计；接着，利用短时傅里叶变换对接收信号处理所形成的频谱瀑布图作为样本，对抗干扰策略网络和信息熵预测网络进行训练；之后，利用信息熵预测网络对抗干扰策略网络的训练样本进行精细化筛选，以提高训练样本的质量，最终提高抗干扰策略的在线决策能力和泛化性能。仿真结果表明，在干扰方干扰策略更新频率不超过通信方40倍且最大干扰通道数为3的极端条件下，基于样本信息熵辅助的通信抗干扰策略仍可取得至少61%的成功率；同时，与其他几种对比抗干扰策略相比，所提通信抗干扰策略具有更快的收敛速度。

Abstract

For the deep reinforcement learning (DRL)-empowered intelligent jamming

an anti-jamming strategy aided by sample information entropy was proposed. Firstly

the anti-jamming strategy network and entropy prediction network were designed based on neural networks. Then

the anti-jamming strategy network and entropy prediction network were trained with the samples of the spectrum waterfall

which were formed by performing the short-time Fourier transform to the received signals. The information entropy prediction network was utilized for fine-grained selection of training samples of the anti-jamming strategy network to improve the quality of training samples

thereby enhancing the ultimate online decision-making capability and generalization performance of the anti-jamming strategy. The simulation results indicate that under the extreme condition where the jamming strategy update frequency does not exceed forty times that of the communication anti-jamming strategy and the maximum number of jamming channels is 3

the proposed anti-jamming strategy

aided by sample information entropy

can still achieve a success rate of at least 61%. Moreover

compared to several other anti-jamming strategies

the proposed strategy demonstrates faster convergence.

关键词

Keywords

references

AMURU S , TEKIN C , VAN DER SCHAAR M , et al . Jamming bandits—a novel learning method for optimal jamming [J ] . IEEE Transactions on Wireless Communications , 2016 , 15 ( 4 ): 2792 - 2808 .

PU Z M , NIU Y T , ZHANG G L . A multi-parameter intelligent communication anti-jamming method based on three-dimensional Q-learning [C ] // Proceedings of the 2022 IEEE 2nd International Conference on Computer Communication and Artificial Intelligence (CCAI) . Piscataway : IEEE Press , 2022 : 205 - 210 .

ZHANG Z X , WU Q H , ZHANG B , et al . Intelligent anti-jamming relay communication system based on reinforcement learning [C ] // Proceedings of the 2019 2nd International Conference on Communication Engineering and Technology (ICCET) . Piscataway : IEEE Press , 2019 : 52 - 56 .

YAO F Q , JIA L L . A collaborative multi-agent reinforcement learning anti-jamming algorithm in wireless networks [J ] . IEEE Wireless Communications Letters , 2019 , 8 ( 4 ): 1024 - 1027 .

ZHANG X B , WANG H , RUAN L , et al . Joint channel, power and bandwidth optimization for Anti-jamming communications: a multi-agent Q-learning approach [C ] // Proceedings of the 2021 13th International Conference on Wireless Communications and Signal Processing (WCSP) . Piscataway : IEEE Press , 2021 : 1 - 6 .

DING Y M , YANG F H , FENG J X , et al . Intelligent Anti-jamming algorithm based on time-frequency domain joint [C ] // Proceedings of the 2021 6th International Symposium on Computer and Information Processing Technology (ISCIPT) . Piscataway : IEEE Press , 2021 : 163 - 167 .

LIU X , XU Y H , JIA L L , et al . Anti-jamming communications using spectrum waterfall: a deep reinforcement learning approach [J ] . IEEE Communications Letters , 2018 , 22 ( 5 ): 998 - 1001 .

LI Y Y , XU Y H , XU Y T , et al . Dynamic spectrum anti-jamming in broadband communications: a hierarchical deep reinforcement learning approach [J ] . IEEE Wireless Communications Letters , 2020 , 9 ( 10 ): 1616 - 1619 .

ZHANG L , MA L , TIAN F , et al . An anti-jamming intelligent decision-making method for multi-user communication based on deep reinforcement learning [C ] // Proceedings of the 2022 IEEE 22nd International Conference on Communication Technology (ICCT) . Piscataway : IEEE Press , 2022 : 1335 - 1339 .

LI W , XU Y H , CHEN J , et al . Know thy enemy: an opponent modeling-based anti-intelligent jamming strategy beyond equilibrium solutions [J ] . IEEE Wireless Communications Letters , 2023 , 12 ( 2 ): 217 - 221 .

SONG B L , XU H , JIANG L , et al . An intelligent decision-making method for anti-jamming communication based on deep reinforcement learning [J ] . Journal of Northwestern Polytechnical University , 2021 , 39 ( 3 ): 641 - 649 .

HAN C , HUO L Y , TONG X H , et al . Spatial anti-jamming scheme for Internet of satellites based on the deep reinforcement learning and stackelberg game [J ] . IEEE Transactions on Vehicular Technology , 2020 , 69 ( 5 ): 5331 - 5342 .

NGUYEN P K H , NGUYEN V H , DO V L . A deep double-Q learning-based scheme for anti-jamming communications [C ] // Proceedings of the 2020 28th European Signal Processing Conference (EUSIPCO) . Piscataway : IEEE Press , 2021 : 1566 - 1570 .

LI Y Y , XU Y H , LI G X , et al . Dynamic spectrum anti-jamming access with fast convergence: a labeled deep reinforcement learning approach [J ] . IEEE Transactions on Information Forensics and Security , 2023 , 18 : 5447 - 5458 .

HAN H , WANG X M , GU F L , et al . Better late than never: GAN-enhanced dynamic anti-jamming spectrum access with incomplete sensing information [J ] . IEEE Wireless Communications Letters , 2021 , 10 ( 8 ): 1800 - 1804 .

CHEN M J , LIU W , ZHANG N , et al . GPDS: a multi-agent deep reinforcement learning game for anti-jamming secure computing in MEC network [J ] . Expert Systems with Applications , 2022 , 210 : 118394 .

冯智斌 , 徐煜华 , 杜智勇 , 等 . 对抗智能干扰的主动防御技术 [J ] . 通信学报 , 2022 , 43 ( 10 ): 42 - 54 .

FENG Z B , XU Y H , DU Z Y , et al . Active defense technology against intelligent jammer [J ] . Journal on Communications , 2022 , 43 ( 10 ): 42 - 54 .

HAN H , LI W , FENG Z B , et al . Proceed from known to unknown: jamming pattern recognition under open-set setting [J ] . IEEE Wireless Communications Letters , 2022 , 11 ( 4 ): 693 - 697 .

NOORI H , SADEGHI VILNI S . Jamming and anti-jamming in interference channels: a stochastic game approach [J ] . IET Communications , 2020 , 14 ( 4 ): 682 - 692 .

浏览量

1135

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于DDPG强化学习的多集群算力资源调度算法

智能反射表面辅助的车载边缘任务卸载和资源分配策略

数字孪生辅助下基于D3QN的车载网络协同卸载算法

基于深度强化学习的分离式数据中心存储资源调度优化方法

动态场景下自动驾驶运行时安全保障的自适应方法