移动车载边缘网络中基于递归深度强化学习的协作缓存接力算法

吴红海; 王白冰; 马华红; 邢玲

doi:10.11959/j.issn.1000-436x.2024195

您当前的位置：

首页 >

文章列表页 >

移动车载边缘网络中基于递归深度强化学习的协作缓存接力算法

学术通信 | 更新时间：2024-12-24

- 移动车载边缘网络中基于递归深度强化学习的协作缓存接力算法
- Recursive deep reinforcement learning-based collaborative caching relay algorithm in mobile vehicular edge network
- 通信学报 2024年45卷第11期页码：277-286
- 作者机构：
  
  河南科技大学信息工程学院，河南洛阳 471023
- 作者简介：
  
  [ "吴红海（1979- ），男，河南洛阳人，博士，河南科技大学副教授，主要研究方向为移动多媒体计算、移动边缘计算。" ]
  [ "王白冰（1998- ），女，河南南阳人，河南科技大学硕士生，主要研究方向为移动边缘缓存。" ]
  [ "马华红（1979- ），女，河南洛阳人，博士，河南科技大学副教授，主要研究方向为人群传感网络、物联网。" ]
  [ "邢玲（1978- ），女，河南洛阳人，博士，河南科技大学教授，主要研究方向为智能信息处理、信息语义分析、多媒体计算与网络智能信息处理、信息语义分析、多媒体计算与网络。" ]
- 基金信息：
  
  国家自然科学基金资助项目(62272146;62071170;62171180;62072158;U23A20272;U22A2069)
- DOI：10.11959/j.issn.1000-436x.2024195
  中图分类号： TN92
- 收稿日期：2024-07-05，
  
  修回日期：2024-10-22，
  
  纸质出版日期：2024-11-25
- 稿件说明：
移动端阅览
吴红海,王白冰,马华红等.移动车载边缘网络中基于递归深度强化学习的协作缓存接力算法[J].通信学报,2024,45(11):277-286.

WU Honghai,WANG Baibing,MA Huahong,et al.Recursive deep reinforcement learning-based collaborative caching relay algorithm in mobile vehicular edge network[J].Journal on Communications,2024,45(11):277-286.
吴红海,王白冰,马华红等.移动车载边缘网络中基于递归深度强化学习的协作缓存接力算法[J].通信学报,2024,45(11):277-286. DOI： 10.11959/j.issn.1000-436x.2024195.

WU Honghai,WANG Baibing,MA Huahong,et al.Recursive deep reinforcement learning-based collaborative caching relay algorithm in mobile vehicular edge network[J].Journal on Communications,2024,45(11):277-286. DOI： 10.11959/j.issn.1000-436x.2024195.

摘要

考虑无路侧单元覆盖的场景，充分利用车辆之间的协作来构建缓存系统，提出一种基于递归深度强化学习的协作缓存接力算法。考虑缓存决策的动态特性，将问题建模为部分可观察的马尔可夫决策过程，利用图神经网络预测车辆轨迹，并通过计算车辆间的连接稳定性度量，选择可作为缓存节点的车辆。此外，将长短期记忆网络嵌入深度确定性策略梯度算法中，以实现最终的缓存决策。仿真结果表明，所提算法在缓存命中率和时延方面优于传统缓存算法。

Abstract

Considering scenarios without road side unit coverage

a recursive deep reinforcement learning-based collaborative caching relay algorithm was proposed to construct a caching system by leveraging the cooperation among vehicles. Recognizing the dynamic nature of caching decisions

the problem was modeled as a partially observable Markov decision process. Vehicle trajectories were predicted using graph neural network

and the connectivity stability between vehicles was measured to select those that could serve as caching nodes. In addition

long short-term memory network was integrated into the deep deterministic policy gradient algorithm to achieve the final caching decision. Simulation results demonstrate that the proposed algorithm outperforms traditional caching algorithms in terms of cache hit ratio and latency.

关键词

Keywords

references

刘雷 , 陈晨 , 冯杰 , 等 . 车载边缘计算中任务卸载和服务缓存的联合智能优化 [J ] . 通信学报 , 2021 , 42 ( 1 ): 18 - 26 .

LIU L , CHEN C , FENG J , et al . Joint intelligent optimization of task offloading and service caching for vehicular edge computing [J ] . Journal on Communications , 2021 , 42 ( 1 ): 18 - 26 .

LU Z J , QU G , LIU Z L . A survey on recent advances in vehicular network security, trust, and privacy [J ] . IEEE Transactions on Intelligent Transportation Systems , 2019 , 20 ( 2 ): 760 - 776 .

SHOJAFAR M , CORDESCHI N , BACCARELLI E . Energy-efficient adaptive resource management for real-time vehicular cloud services [J ] . IEEE Transactions on Cloud Computing , 2019 , 7 ( 1 ): 196 - 209 .

ZHU D J , DU H W , SUN Y D , et al . Massive files prefetching model based on LSTM neural network with cache transaction strategy [J ] . Computers , Materials & Continua, 2020 , 63 ( 2 ): 979 - 993 .

SU Z , HUI Y L , XU Q C , et al . An edge caching scheme to distribute content in vehicular networks [J ] . IEEE Transactions on Vehicular Technology , 2018 , 67 ( 6 ): 5346 - 5356 .

TAN L T , HU R Q , HANZO L . Twin-timescale artificial intelligence aided mobility-aware edge caching and computing in vehicular networks [J ] . IEEE Transactions on Vehicular Technology , 2019 , 68 ( 4 ): 3086 - 3099 .

MUSA S S , ZENNARO M , LIBSIE M , et al . Mobility-aware proactive edge caching optimization scheme in information-centric IoV networks [J ] . Sensors , 2022 , 22 ( 4 ): 1387 .

SU Z , HUI Y L , GUO S . D2D-based content delivery with parked vehicles in vehicular social networks [J ] . IEEE Wireless Communications , 2016 , 23 ( 4 ): 90 - 95 .

ZHAO W C , QIN Y J , GAO D Y , et al . An efficient cache strategy in information centric networking vehicle-to-vehicle scenario [J ] . IEEE Access , 2017 , 5 : 12657 - 12667 .

JAVED M A , ZEADALLY S . AI-empowered content caching in vehicular edge computing: opportunities and challenges [J ] . IEEE Network , 2021 , 35 ( 3 ): 109 - 115 .

KHANAL S , THAR K , HUH E N . DCoL: distributed collaborative learning for proactive content caching at edge networks [J ] . IEEE Access , 2021 , 9 : 73495 - 73505 .

HU Z W , ZHENG Z J , WANG T , et al . Roadside unit caching: auction-based storage allocation for multiple content providers [J ] . IEEE Transactions on Wireless Communications , 2017 , 16 ( 10 ): 6321 - 6334 .

BITAGHSIR S A , DADLANI A , BORHANI M , et al . Multi-armed bandit learning for cache content placement in vehicular social networks [J ] . IEEE Communications Letters , 2019 , 23 ( 12 ): 2321 - 2324 .

ZHAO Z L , GUARDALBEN L , KARIMZADEH M , et al . Mobility prediction-assisted over-the-top edge prefetching for hierarchical VANETs [J ] . IEEE Journal on Selected Areas in Communications , 2018 , 36 ( 8 ): 1786 - 1801 .

QIAO G H , LENG S P , MAHARJAN S , et al . Deep reinforcement learning for cooperative content caching in vehicular edge computing and networks [J ] . IEEE Internet of Things Journal , 2020 , 7 ( 1 ): 247 - 257 .

ZHANG Y , LI C L , LUAN T H , et al . A mobility-aware vehicular caching scheme in content centric networks: model and optimization [J ] . IEEE Transactions on Vehicular Technology , 2019 , 68 ( 4 ): 3100 - 3112 .

HU B B , FANG L Y , CHENG X , et al . In-vehicle caching (IV-cache) via dynamic distributed storage relay (D 2 SR) in vehicular networks [J ] . IEEE Transactions on Vehicular Technology , 2019 , 68 ( 1 ): 843 - 855 .

WANG R Y , KAN Z W , CUI Y P , et al . Cooperative caching strategy with content request prediction in Internet of vehicles [J ] . IEEE Internet of Things Journal , 2021 , 8 ( 11 ): 8964 - 8975 .

XIONG X , ZHENG K , LEI L , et al . Resource allocation based on deep reinforcement learning in IoT edge computing [J ] . IEEE Journal on Selected Areas in Communications , 2020 , 38 ( 6 ): 1133 - 1146 .

SONG J J , SHENG M , QUEK T Q S , et al . Learning-based content caching and sharing for wireless networks [J ] . IEEE Transactions on Communications , 2017 , 65 ( 10 ): 4309 - 4324 .

ZHAO J J , LIU Y W , CHAI K K , et al . Joint subchannel and power allocation for NOMA enhanced D2D communications [J ] . IEEE Transactions on Communications , 2017 , 65 ( 11 ): 5081 - 5094 .

XU W P , QIU R H , JIANG X Q . Resource allocation in heterogeneous cognitive radio network with non-orthogonal multiple access [J ] . IEEE Access , 2019 , 7 : 57488 - 57499 .

HUANG X Y , WANG P , CHENG X J , et al . The Apolloscape open dataset for autonomous driving and its application [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2020 , 42 ( 10 ): 2702 - 2719 .

MAHANANDA I G E , YOVITA L V , NEGARA R M . Performance of homogeneous and heterogeneous cache policy for named data network [C ] // Proceedings of the 2022 10th International Conference on Information and Communication Technology (ICoICT) . Piscataway : IEEE Press , 2022 : 120 - 123 .

ZHOU H , WU T , ZHANG H J , et al . Incentive-driven deep reinforcement learning for content caching and D2D offloading [J ] . IEEE Journal on Selected Areas in Communications , 2021 , 39 ( 8 ): 2445 - 2460 .

LIU Y , MAO Y L , SHANG X J , et al . Distributed cooperative caching in unreliable edge environments [C ] // Proceedings of the IEEE INFOCOM 2022 - IEEE Conference on Computer Communications . Piscataway : IEEE Press , 2022 : 1049 - 1058 .

LIU X , XU S Y , YANG C , et al . Deep reinforcement learning empowered edge collaborative caching scheme for Internet of vehicles [J ] . Computer Systems Science and Engineering , 2022 , 42 ( 1 ): 271 - 287 .

SETHUMURUGAN S , YIN J M , SARTORI J . Designing a cost-effective cache replacement policy using machine learning [C ] // Proceedings of the 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA) . Piscataway : IEEE Press , 2021 : 291 - 303 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于马尔可夫决策的理性秘密共享方案