Deep reinforcement learning-empowered anti-jamming strategy aided by sample information entropy

LI Gang; WU Qi; WANG Xiang; LUO Hao; LI Lianghong; JING Xiaorong; CHEN Qianbin

doi:10.11959/j.issn.1000-436x.2024161

您当前的位置：

首页 >

文章列表页 >

Deep reinforcement learning-empowered anti-jamming strategy aided by sample information entropy

Papers | 更新时间：2024-10-10

- Deep reinforcement learning-empowered anti-jamming strategy aided by sample information entropy
- Journal on Communications Vol. 45, Issue 9, Pages: 115-128(2024)
- 作者机构：
  
  1.中国西南电子技术研究所，四川成都 610036
  2.重庆邮电大学通信与信息工程学院，重庆 400065
- 作者简介：
- 基金信息：
  
  China Electronics Tian’ao Innovation Theory and Technology Group Fund(2022-1193-04-04)
- DOI：10.11959/j.issn.1000-436x.2024161
  CLC： TN975
- Received：05 February 2024，
  
  Revised：2024-08-06，
  
  Published：25 September 2024
- 稿件说明：
移动端阅览
李刚,吴麒,王翔等.基于样本信息熵辅助的深度强化学习抗干扰策略[J].通信学报,2024,45(09):115-128.

LI Gang,WU Qi,WANG Xiang,et al.Deep reinforcement learning-empowered anti-jamming strategy aided by sample information entropy[J].Journal on Communications,2024,45(09):115-128.
李刚,吴麒,王翔等.基于样本信息熵辅助的深度强化学习抗干扰策略[J].通信学报,2024,45(09):115-128. DOI： 10.11959/j.issn.1000-436x.2024161.

LI Gang,WU Qi,WANG Xiang,et al.Deep reinforcement learning-empowered anti-jamming strategy aided by sample information entropy[J].Journal on Communications,2024,45(09):115-128. DOI： 10.11959/j.issn.1000-436x.2024161.

摘要

针对深度强化学习驱动的智能化干扰，提出了一种基于样本信息熵辅助的通信抗干扰策略。首先，基于神经网络对抗干扰策略网络和熵预测网络进行设计；接着，利用短时傅里叶变换对接收信号处理所形成的频谱瀑布图作为样本，对抗干扰策略网络和信息熵预测网络进行训练；之后，利用信息熵预测网络对抗干扰策略网络的训练样本进行精细化筛选，以提高训练样本的质量，最终提高抗干扰策略的在线决策能力和泛化性能。仿真结果表明，在干扰方干扰策略更新频率不超过通信方40倍且最大干扰通道数为3的极端条件下，基于样本信息熵辅助的通信抗干扰策略仍可取得至少61%的成功率；同时，与其他几种对比抗干扰策略相比，所提通信抗干扰策略具有更快的收敛速度。

Abstract

For the deep reinforcement learning (DRL)-empowered intelligent jamming

an anti-jamming strategy aided by sample information entropy was proposed. Firstly

the anti-jamming strategy network and entropy prediction network were designed based on neural networks. Then

the anti-jamming strategy network and entropy prediction network were trained with the samples of the spectrum waterfall

which were formed by performing the short-time Fourier transform to the received signals. The information entropy prediction network was utilized for fine-grained selection of training samples of the anti-jamming strategy network to improve the quality of training samples

thereby enhancing the ultimate online decision-making capability and generalization performance of the anti-jamming strategy. The simulation results indicate that under the extreme condition where the jamming strategy update frequency does not exceed forty times that of the communication anti-jamming strategy and the maximum number of jamming channels is 3

the proposed anti-jamming strategy

aided by sample information entropy

can still achieve a success rate of at least 61%. Moreover

compared to several other anti-jamming strategies

the proposed strategy demonstrates faster convergence.

关键词

Keywords

references

AMURU S , TEKIN C , VAN DER SCHAAR M , et al . Jamming bandits—a novel learning method for optimal jamming [J ] . IEEE Transactions on Wireless Communications , 2016 , 15 ( 4 ): 2792 - 2808 .

PU Z M , NIU Y T , ZHANG G L . A multi-parameter intelligent communication anti-jamming method based on three-dimensional Q-learning [C ] // Proceedings of the 2022 IEEE 2nd International Conference on Computer Communication and Artificial Intelligence (CCAI) . Piscataway : IEEE Press , 2022 : 205 - 210 .

ZHANG Z X , WU Q H , ZHANG B , et al . Intelligent anti-jamming relay communication system based on reinforcement learning [C ] // Proceedings of the 2019 2nd International Conference on Communication Engineering and Technology (ICCET) . Piscataway : IEEE Press , 2019 : 52 - 56 .

YAO F Q , JIA L L . A collaborative multi-agent reinforcement learning anti-jamming algorithm in wireless networks [J ] . IEEE Wireless Communications Letters , 2019 , 8 ( 4 ): 1024 - 1027 .

ZHANG X B , WANG H , RUAN L , et al . Joint channel, power and bandwidth optimization for Anti-jamming communications: a multi-agent Q-learning approach [C ] // Proceedings of the 2021 13th International Conference on Wireless Communications and Signal Processing (WCSP) . Piscataway : IEEE Press , 2021 : 1 - 6 .

DING Y M , YANG F H , FENG J X , et al . Intelligent Anti-jamming algorithm based on time-frequency domain joint [C ] // Proceedings of the 2021 6th International Symposium on Computer and Information Processing Technology (ISCIPT) . Piscataway : IEEE Press , 2021 : 163 - 167 .

LIU X , XU Y H , JIA L L , et al . Anti-jamming communications using spectrum waterfall: a deep reinforcement learning approach [J ] . IEEE Communications Letters , 2018 , 22 ( 5 ): 998 - 1001 .

LI Y Y , XU Y H , XU Y T , et al . Dynamic spectrum anti-jamming in broadband communications: a hierarchical deep reinforcement learning approach [J ] . IEEE Wireless Communications Letters , 2020 , 9 ( 10 ): 1616 - 1619 .

ZHANG L , MA L , TIAN F , et al . An anti-jamming intelligent decision-making method for multi-user communication based on deep reinforcement learning [C ] // Proceedings of the 2022 IEEE 22nd International Conference on Communication Technology (ICCT) . Piscataway : IEEE Press , 2022 : 1335 - 1339 .

LI W , XU Y H , CHEN J , et al . Know thy enemy: an opponent modeling-based anti-intelligent jamming strategy beyond equilibrium solutions [J ] . IEEE Wireless Communications Letters , 2023 , 12 ( 2 ): 217 - 221 .

SONG B L , XU H , JIANG L , et al . An intelligent decision-making method for anti-jamming communication based on deep reinforcement learning [J ] . Journal of Northwestern Polytechnical University , 2021 , 39 ( 3 ): 641 - 649 .

HAN C , HUO L Y , TONG X H , et al . Spatial anti-jamming scheme for Internet of satellites based on the deep reinforcement learning and stackelberg game [J ] . IEEE Transactions on Vehicular Technology , 2020 , 69 ( 5 ): 5331 - 5342 .

NGUYEN P K H , NGUYEN V H , DO V L . A deep double-Q learning-based scheme for anti-jamming communications [C ] // Proceedings of the 2020 28th European Signal Processing Conference (EUSIPCO) . Piscataway : IEEE Press , 2021 : 1566 - 1570 .

LI Y Y , XU Y H , LI G X , et al . Dynamic spectrum anti-jamming access with fast convergence: a labeled deep reinforcement learning approach [J ] . IEEE Transactions on Information Forensics and Security , 2023 , 18 : 5447 - 5458 .

HAN H , WANG X M , GU F L , et al . Better late than never: GAN-enhanced dynamic anti-jamming spectrum access with incomplete sensing information [J ] . IEEE Wireless Communications Letters , 2021 , 10 ( 8 ): 1800 - 1804 .

CHEN M J , LIU W , ZHANG N , et al . GPDS: a multi-agent deep reinforcement learning game for anti-jamming secure computing in MEC network [J ] . Expert Systems with Applications , 2022 , 210 : 118394 .

冯智斌 , 徐煜华 , 杜智勇 , 等 . 对抗智能干扰的主动防御技术 [J ] . 通信学报 , 2022 , 43 ( 10 ): 42 - 54 .

FENG Z B , XU Y H , DU Z Y , et al . Active defense technology against intelligent jammer [J ] . Journal on Communications , 2022 , 43 ( 10 ): 42 - 54 .

HAN H , LI W , FENG Z B , et al . Proceed from known to unknown: jamming pattern recognition under open-set setting [J ] . IEEE Wireless Communications Letters , 2022 , 11 ( 4 ): 693 - 697 .

NOORI H , SADEGHI VILNI S . Jamming and anti-jamming in interference channels: a stochastic game approach [J ] . IET Communications , 2020 , 14 ( 4 ): 682 - 692 .

Views

1333

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

GAT-based decision mechanism for decentralized joint routing and spectrum access

Multi-cluster computing power resource scheduling algorithm based on DDPG reinforcement learning

Task offloading and resource allocation strategy for vehicular edge computing assisted by intelligent reflecting surfaces

D3QN-based collaborative offloading algorithm for vehicular networks assisted by digital twins

Storage resource scheduling optimization method for separated data center based on deep reinforcement learning

Related Author

Zhou Zibo

Ren Baoquan

Zhong Xudong

Liu Qi

Qin Zhen

HU Yahui

WANG Yuelin

ZHANG Chenkang

Related Institution

Systems Engineering Institute, Academy of Military Science

Air Force Early Warning Academy

School of Artificial Intelligence, China University of Mining and Technology-Beijing

Computer Network information Center, Chinese Academy of Sciences

CHN Energy Zhi Shen Control Technology Co., Ltd.

AI问答

⁰