基于深度强化学习的OFDM自适应导频设计

刘乔寿; 周雄; 刘爽; 邓义锋

doi:10.11959/j.issn.1000-436x.2023169

您当前的位置：

首页 >

文章列表页 >

基于深度强化学习的OFDM自适应导频设计

学术论文 | 更新时间：2024-06-06

- 基于深度强化学习的OFDM自适应导频设计
- Adaptive pilot design for OFDM based on deep reinforcement learning
- 通信学报 2023年44卷第9期页码：104-114
- 作者机构：
  
  1. 重庆邮电大学通信与信息工程学院，重庆 400065
  2. 先进网络与智能互联技术重庆市高校重点实验室，重庆 400065
  3. 泛在感知与互联重庆市重点实验室，重庆 400065
- 作者简介：
  
  [ "刘乔寿（1979- ），男，云南曲靖人，博士，重庆邮电大学副教授、硕士生导师，主要研究方向为5G超密集网络干扰协调、云边协同智能计算、FPGA智能算法加速、物联网系统及终端设备开发" ]
  [ "周雄（1999- ），男，湖北荆州人，重庆邮电大学硕士生，主要研究方向为基于深度强化学习的信道估计" ]
  [ "刘爽（2000- ），男，安徽桐城人，重庆邮电大学硕士生，主要研究方向为空中计算" ]
  [ "邓义锋（1999- ），男，四川成都人，重庆邮电大学硕士生，主要研究方向为空中计算" ]
- 基金信息：
  
  国家自然科学基金资助项目(61901075);重庆市教委科学技术基金资助项目(KJZDK202200604)
- DOI：10.11959/j.issn.1000-436x.2023169
  中图分类号： TN92
- 网络首发：2023-09，
  
  纸质出版：2023-09-25
- 稿件说明：
移动端阅览
刘乔寿, 周雄, 刘爽, 等. 基于深度强化学习的OFDM自适应导频设计[J]. 通信学报, 2023,44(9):104-114.

Qiaoshou LIU, Xiong ZHOU, Shuang LIU, et al. Adaptive pilot design for OFDM based on deep reinforcement learning[J]. Journal on Communications, 2023, 44(9): 104-114.
刘乔寿, 周雄, 刘爽, 等. 基于深度强化学习的OFDM自适应导频设计[J]. 通信学报, 2023,44(9):104-114. DOI： 10.11959/j.issn.1000-436x.2023169.

Qiaoshou LIU, Xiong ZHOU, Shuang LIU, et al. Adaptive pilot design for OFDM based on deep reinforcement learning[J]. Journal on Communications, 2023, 44(9): 104-114. DOI： 10.11959/j.issn.1000-436x.2023169.

摘要

针对正交频分复用系统，提出了一种基于深度强化学习的自适应导频设计算法。将导频设计问题映射为马尔可夫决策过程，导频位置的索引定义为动作，用基于减少均方误差的策略定义奖励函数，使用深度强化学习来更新导频位置。根据信道条件自适应地动态分配导频，从而利用信道特性对抗信道衰落。仿真结果表明，所提算法在3GPP的3种典型多径信道下相较于传统导频均匀分配方案信道估计性能有显著的提升。

Abstract

For orthogonal frequency division multiplexing (OFDM) systems

an adaptive pilot design algorithm based on deep reinforcement learning was proposed.The pilot design problem was formulated as a Markov decision process

where the index of pilot positions was defined as actions.A reward function based on mean squared error (MSE) reduction strategy was formulated

and deep reinforcement learning was employed to update the pilot positions.The pilot was adaptively and dynamically allocated based on channel conditions

thereby utilizing channel characteristics to combat channel fading.The simulation results show that the proposed algorithm has significantly improved channel estimation performance compared with the traditional pilot uniform allocation scheme under three typical multipath channels of 3GPP.

关键词

Keywords

references

SOLTANI M , POURAHMADI V , SHEIKHZADEH H . Pilot pattern design for deep learning-based channel estimation in OFDM systems [J ] . IEEE Wireless Communications Letters , 2020 , 9 ( 12 ): 2173 - 2176 .

MASHHADI M B , GÜNDÜZ D . Pruning the pilots:deep learning-based pilot design and channel estimation for MIMO-OFDM systems [J ] . IEEE Transactions on Wireless Communications , 2021 , 20 ( 10 ): 6315 - 6328 .

CHEN H , ZHANG Q Q , LONG R Z , et al . Pilot design and signal detection for symbiotic radio over OFDM carriers [C ] // Proceedings of IEEE Global Communications Conference . Piscataway:IEEE Press , 2023 : 1887 - 1892 .

CAO J , ZHU X , JIANG Y F , et al . Independent pilots versus shared pilots:short frame structure optimization for heterogeneous-traffic URLLC networks [J ] . IEEE Transactions on Wireless Communications , 2022 , 21 ( 8 ): 5755 - 5769 .

LIN X , LIU A J , HAN C , et al . Joint pilot spacing and power optimization scheme for nonstationary wireless channel:a deep reinforcement learning approach [J ] . IEEE Wireless Communications Letters , 2023 , 12 ( 3 ): 540 - 544 .

RAO R M , MAROJEVIC V , REED J H . Adaptive pilot patterns for CA-OFDM systems in nonstationary wireless channels [J ] . IEEE Transactions on Vehicular Technology , 2018 , 67 ( 2 ): 1231 - 1244 .

LI F , SHEN B W , GUO J L , et al . Dynamic spectrum access for Internet-of-things based on federated deep reinforcement learning [J ] . IEEE Transactions on Vehicular Technology , 2022 , 71 ( 7 ): 7952 - 7956 .

CHEN M , LIU A , LIU W , et al . RDRL:a recurrent deep reinforcement learning scheme for dynamic spectrum access in reconfigurable wireless networks [J ] . IEEE Transactions on Network Science and Engineering , 2021 , 9 ( 2 ): 364 - 376 .

HAN H , XU Y , JIN z , et al . primary-user-friendly dynamic spectrum anti-jamming access:a GAN-enhanced deep reinforcement learning Approach [J ] . IEEE Wireless Communications Letters , 2021 , 11 ( 2 ): 258 - 262 .

LI J , GAO H , LV T , et al . Deep reinforcement learning based computation offloading and resource allocation for MEC [C ] // Proceedings of IEEE Wireless Communications and Networking Conference (WCNC) . Piscataway:IEEE Press , 2018 : 1 - 6 .

YANG H , WEI Z , FENG Z , et al . Intelligent computation offloading for MEC-based cooperative vehicle infrastructure system:a deep reinforcement learning approach [J ] . IEEE Transactions on Vehicular Technology , 2022 , 71 ( 7 ): 7665 - 7679 .

WANG J , ZHAO L , LIU J , et al . Smart resource allocation for mobile edge computing:a deep reinforcement learning approach [J ] . IEEE Transactions on Emerging Topics in Computing , 2019 , 9 ( 3 ): 1529 - 1541 .

NGUYEN T T , REDDI V J . Deep reinforcement learning for cyber security [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2023 , 34 ( 8 ): 3779 - 3795 .

ZHANG Y , MOU Z , GAO F , et al . UAV-enabled secure communications by multi-agent deep reinforcement learning [J ] . IEEE Transactions on Vehicular Technology , 2020 , 69 ( 10 ): 11599 - 11611 .

OH M S , HOSSEINALIPOUR S , KIM T , et al . Channel estimation via successive denoising in MIMO OFDM systems:a reinforcement learning approach [C ] // Proceedings of IEEE International Conference on Communications (ICC) . Piscataway:IEEE Press , 2021 : 1 - 6 .

CHU M , LIU A , LAU V K N , et al . Deep reinforcement learning based end-to-end multiuser channel prediction and beamforming [J ] . IEEE Transactions on Wireless Communications , 2022 , 21 ( 12 ): 10271 - 10285 .

HUANG R , WONG V W S , SCHOBER R . Throughput optimization for grant-free multiple access with multiagent deep reinforcement learning [J ] . IEEE Transactions on Wireless Communications , 2020 , 20 ( 1 ): 228 - 242 .

WANG Z , SCHAUL T , HESSEL M , et al . Dueling network architectures for deep reinforcement learning [C ] // Proceedings of International Conference on Machine Learning . New York:PMLR , 2016 : 1995 - 2003 .

European Telecommunications Standards Institute . 3GPP:TS36.104 [S ] .[2023-05-06 ] .

浏览量

697

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于DDPG强化学习的多集群算力资源调度算法

智能反射表面辅助的车载边缘任务卸载和资源分配策略

数字孪生辅助下基于D3QN的车载网络协同卸载算法

基于深度强化学习的分离式数据中心存储资源调度优化方法

动态场景下自动驾驶运行时安全保障的自适应方法