内容新鲜度保障的车联网多智能体缓存分发策略

崔亚平; 石宏吉; 吴大鹏; 何鹏; 王汝言

doi:10.11959/j.issn.1000-436x.2025013

您当前的位置：

首页 >

文章列表页 >

内容新鲜度保障的车联网多智能体缓存分发策略

学术论文 | 更新时间：2025-02-13

- 内容新鲜度保障的车联网多智能体缓存分发策略
- Multi-agent caching distribution strategy for content freshness guarantee in IoV
- 通信学报 2025年46卷第1期页码：52-66
- 作者机构：
  
  1.重庆邮电大学通信与信息工程学院，重庆 400065
  2.先进网络与智能互联技术重庆市高校重点实验室，重庆 400065
  3.泛在感知与互联重庆市重点实验室，重庆 400065
- 作者简介：
  
  [ "崔亚平（1986- ），男，河南新乡人，博士，重庆邮电大学副教授，主要研究方向为智能反射面、移动边缘计算和车联网的网络功能虚拟化等。" ]
  [ "石宏吉（1998- ），男，四川邻水人，重庆邮电大学硕士生，主要研究方向为车载网络和边缘缓存等。" ]
  [ "吴大鹏（1979- ），男，黑龙江大庆人，博士，重庆邮电大学教授、博士生导师，主要研究方向为普适网络、IP QoS架构、网络可靠性以及通信系统的性能评估等。" ]
  [ "何鹏（1990- ），男，重庆人，博士，重庆邮电大学副教授，主要研究方向为移动边缘计算、分子通信。" ]
  [ "王汝言（1969- ），男，湖北浠水人，博士，重庆邮电大学教授、博士生导师，主要研究方向为泛在网络、多媒体信息处理等。" ]
- 基金信息：
  
  国家自然科学基金资助项目(61801065;62271096;61871062;U20A20157;62061007);重庆市教委科学技术研究基金资助项目(KJQN202000603;KJQN202300621);重庆市自然科学基金资助项目(CSTB2022NSCQ-MSX0468;cstc2020jcyjzdxmX0024;cstc2021jcyjmsxmX0892;CSTB2023NSCQ-LZX0134);重庆市高校创新研究群体基金资助项目;The University Innovation Research Group of Chongqing(CXQT20017);重邮信通青创团队支持计划基金资助项目;The Youth Innovation Group Support Program of ICE Discipline of CQUPT(SCIE-QN-2022-04);四川省重点研发计划基金资助项目(2024YFHZ0093)
- DOI：10.11959/j.issn.1000-436x.2025013
  中图分类号： TN927
- 收稿日期：2024-09-25，
  
  修回日期：2024-12-10，
  
  纸质出版日期：2025-01-25
- 稿件说明：
移动端阅览
崔亚平,石宏吉,吴大鹏等.内容新鲜度保障的车联网多智能体缓存分发策略[J].通信学报,2025,46(01):52-66.

CUI Yaping,SHI Hongji,WU Dapeng,et al.Multi-agent caching distribution strategy for content freshness guarantee in IoV[J].Journal on Communications,2025,46(01):52-66.
崔亚平,石宏吉,吴大鹏等.内容新鲜度保障的车联网多智能体缓存分发策略[J].通信学报,2025,46(01):52-66. DOI： 10.11959/j.issn.1000-436x.2025013.

CUI Yaping,SHI Hongji,WU Dapeng,et al.Multi-agent caching distribution strategy for content freshness guarantee in IoV[J].Journal on Communications,2025,46(01):52-66. DOI： 10.11959/j.issn.1000-436x.2025013.

摘要

车辆需要频繁动态变化内容支持车联网（IoV）时延敏感型应用，这会增加宏基站（MBS）负载，降低内容新鲜度。利用边缘缓存将最新内容提前缓存在小基站（SBS）能有效降低车辆时延和提高内容新鲜度。对影响时延和内容信息年龄（AoI）进行深入分析，提出一种内容新鲜度保障的多智能体强化学习（MARL）算法，通过优化缓存分发决策保障车辆获得高新鲜度内容。仿真结果表明，所提算法不仅收敛速度更快，而且在降低车辆时延和提升内容新鲜度方面表现出更好效果。

Abstract

Vehicles need to dynamically changing content to support latency-sensitive applications in Internet of vehicles (IoV)

thereby increasing the load on the macro base station (MBS) and reducing the freshness of content. Utilizing edge caching to cache the latest content in small base station (SBS) can effectively reduce the latency and improve the content freshness. An in-depth analysis was conducted on latency and content's age of information (AoI). A content freshness assurance multi-agent reinforcement learning (MARL) algorithm was proposed

which optimized cache distribution decisions to guarantee high freshness. Simulation results show that the proposed algorithm not only converges faster but also demonstrates better performance in reducing latency and enhancing content freshness.

关键词

Keywords

references

NAN Z J , JIA Y J , REN Z , et al . Delay-aware content delivery with deep reinforcement learning in Internet of vehicles [J ] . IEEE Transactions on Intelligent Transportation Systems , 2022 , 23 ( 7 ): 8918 - 8929 .

ZHANG K , LENG S P , HE Y J , et al . Mobile edge computing and networking for green and low-latency Internet of things [J ] . IEEE Communications Magazine , 2018 , 56 ( 5 ): 39 - 45 .

LI Y , XIA S C , ZHENG M Y , et al . Lyapunov optimization-based trade-off policy for mobile cloud offloading in heterogeneous wireless networks [J ] . IEEE Transactions on Cloud Computing , 2022 , 10 ( 1 ): 491 - 505 .

LI Y , LIU J , CAO B , et al . Joint optimization of radio and virtual machine resources with uncertain user demands in mobile cloud computing [J ] . IEEE Transactions on Multimedia , 2018 , 20 ( 9 ): 2427 - 2438 .

JIANG K , SUN C , ZHOU H , et al . Intelligence-empowered mobile edge computing: framework, issues, implementation, and outlook [J ] . IEEE Network , 2021 , 35 ( 5 ): 74 - 82 .

NING Z L , ZHANG K Y , WANG X J , et al . Intelligent edge computing in Internet of vehicles: a joint computation offloading and caching solution [J ] . IEEE Transactions on Intelligent Transportation Systems , 2021 , 22 ( 4 ): 2212 - 2225 .

ZHOU J W , CHEN F F , HE Q , et al . Data caching optimization with fairness in mobile edge computing [J ] . IEEE Transactions on Services Computing , 2023 , 16 ( 3 ): 1750 - 1762 .

WANG R Y , KAN Z W , CUI Y P , et al . Cooperative caching strategy with content request prediction in Internet of vehicles [J ] . IEEE Internet of Things Journal , 2021 , 8 ( 11 ): 8964 - 8975 .

WU H H , FAN Y Z , JIN J C , et al . Social-aware decentralized cooperative caching for Internet of vehicles [J ] . IEEE Internet of Things Journal , 2023 , 10 ( 16 ): 14834 - 14845 .

QIAO G H , LENG S P , MAHARJAN S , et al . Deep reinforcement learning for cooperative content caching in vehicular edge computing and networks [J ] . IEEE Internet of Things Journal , 2020 , 7 ( 1 ): 247 - 257 .

TRAN T X , POMPILI D . Adaptive bitrate video caching and processing in mobile-edge computing networks [J ] . IEEE Transactions on Mobile Computing , 2019 , 18 ( 9 ): 1965 - 1978 .

ZHANG T K , FANG X Y , LIU Y W , et al . D2D-enabled mobile user edge caching: a multi-winner auction approach [J ] . IEEE Transactions on Vehicular Technology , 2019 , 68 ( 12 ): 12314 - 12328 .

HUANG X G , XU K , CHEN Q B , et al . Delay-aware caching in Internet-of-vehicles networks [J ] . IEEE Internet of Things Journal , 2021 , 8 ( 13 ): 10911 - 10921 .

YANG P , LYU F , WU W , et al . Edge coordinated query configuration for low-latency and accurate video analytics [J ] . IEEE Transactions on Industrial Informatics , 2020 , 16 ( 7 ): 4855 - 4864 .

ZHANG M , ARAFA A , HUANG J W , et al . Pricing fresh data [J ] . IEEE Journal on Selected Areas in Communications , 2021 , 39 ( 5 ): 1211 - 1225 .

TIAN H , XU X L , QI L Y , et al . CoPace: edge computation offloading and caching for self-driving with deep reinforcement learning [J ] . IEEE Transactions on Vehicular Technology , 2021 , 70 ( 12 ): 13281 - 13293 .

YATES R D , SUN Y , BROWN D R , et al . Age of information: an introduction and survey [J ] . IEEE Journal on Selected Areas in Communications , 2021 , 39 ( 5 ): 1183 - 1210 .

ZHANG S , WANG L D , LUO H B , et al . Age of information and delay tradeoff with freshness-aware mobile edge cache update [C ] // Proceedings of the 2019 IEEE Global Communications Conference (GLOBECOM) . Piscataway : IEEE Press , 2019 : 1 - 6 .

AHANI G , YUAN D . Optimal content caching and recommendation with age of information [J ] . IEEE Transactions on Mobile Computing , 2024 , 23 ( 1 ): 689 - 704 .

ZHANG S , LI J J , LUO H B , et al . Towards fresh and low-latency content delivery in vehicular networks: an edge caching aspect [C ] // Proceedings of the 2018 10th International Conference on Wireless Communications and Signal Processing (WCSP) . Piscataway : IEEE Press , 2018 : 1 - 6 .

ZHANG S , LI J J , LUO H B , et al . Low-latency and fresh content provision in information-centric vehicular networks [J ] . IEEE Transactions on Mobile Computing , 2022 , 21 ( 5 ): 1723 - 1738 .

AHANI G , YUAN D . Accounting for information freshness in scheduling of content caching [C ] // Proceedings of the ICC 2020 - 2020 IEEE International Conference on Communications (ICC) . Piscataway : IEEE Press , 2020 : 1 - 6 .

王红艳 , 孙其博 , 马骁 , 等 . 边缘辅助实时应用中信息年龄感知的任务调度 [J ] . 通信学报 , 2024 , 45 ( 6 ): 144 - 159 .

WANG H Y , SUN Q B , MA X , et al . AoI-aware task scheduling in edge-assisted real-time applications [J ] . Journal on Communications , 2024 , 45 ( 6 ): 144 - 159 .

张政 , 谢鑫 , 柏桐 , 等 . 信息年龄和能耗联合优化的无线体域网卸载和调度策略 [J ] . 通信学报 , 2024 , 45 ( 9 ): 92 - 100 .

ZHANG Z , XIE X , BAI T , et al . Joint optimization strategy of age of information and energy consumption for offloading and scheduling in WBAN [J ] . Journal on Communications , 2024 , 45 ( 9 ): 92 - 100 .

ZHANG D G , WANG W J , ZHANG J , et al . Novel edge caching approach based on multi-agent deep reinforcement learning for Internet of vehicles [J ] . IEEE Transactions on Intelligent Transportation Systems , 2023 , 24 ( 8 ): 8324 - 8338 .

MA M Y , WONG V W S . Age of information driven cache content update scheduling for dynamic contents in heterogeneous networks [J ] . IEEE Transactions on Wireless Communications , 2020 , 19 ( 12 ): 8427 - 8441 .

XU C , WANG X J , YANG H H , et al . AoI and energy consumption oriented dynamic status updating in caching enabled IoT networks [C ] // Proceedings of the IEEE INFOCOM 2020-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS) . Piscataway : IEEE Press , 2020 : 710 - 715 .

WU X W , LI X H , LI J , et al . Caching transient content for IoT sensing: multi-agent soft actor-critic [J ] . IEEE Transactions on Communications , 2021 , 69 ( 9 ): 5886 - 5901 .

LIANG L , YE H , LI G Y . Spectrum sharing in vehicular networks based on multi-agent reinforcement learning [J ] . IEEE Journal on Selected Areas in Communications , 2019 , 37 ( 10 ): 2282 - 2292 .

LIU Y , YU H M , XIE S L , et al . Deep reinforcement learning for offloading and resource allocation in vehicle edge computing and networks [J ] . IEEE Transactions on Vehicular Technology , 2019 , 68 ( 11 ): 11158 - 11168 .

ZHANG S , WANG L D , LUO H B , et al . AoI-delay tradeoff in mobile edge caching with freshness-aware content refreshing [J ] . IEEE Transactions on Wireless Communications , 2021 , 20 ( 8 ): 5329 - 5342 .

CHU T S , WANG J , CODECÀ L , et al . Multi-agent deep reinforcement learning for large-scale traffic signal control [J ] . IEEE Transactions on Intelligent Transportation Systems , 2020 , 21 ( 3 ): 1086 - 1095 .

IQBAL S , SHA F . Actor-attention-critic for multi-agent reinforcement learning [C ] // Proceedings of the 36th International Conference on Machine Learning . Saarland : DBLP , 2020 : 2961 - 2970 .

ZHAO Y W , LI R B , WANG C Y , et al . Neighboring-aware caching in heterogeneous edge networks by actor-attention-critic learning [C ] // Proceedings of the ICC 2021-IEEE International Conference on Communications . Piscataway : IEEE Press , 2021 : 1 - 6 .

CHEN S Z , HU J L , SHI Y , et al . Vehicle-to-everything (V2X) services supported by LTE-based systems and 5G [J ] . IEEE Communications Standards Magazine , 2017 , 1 ( 2 ): 70 - 76 .

HE P , CAO L , CUI Y P , et al . Multi-agent caching strategy for spatial-temporal popularity in IoV [J ] . IEEE Transactions on Vehicular Technology , 2023 , 72 ( 10 ): 13536 - 13546 .

ZHI Y , TIAN J , LIU Q Q , et al . Multi-agent reinforcement learning for cooperative edge caching in heterogeneous networks [C ] // Proceedings of the 2021 13th International Conference on Wireless Communications and Signal Processing (WCSP) . Piscataway : IEEE Press , 2021 : 1 - 6 .

ZHOU X H , BILAL M , DOU R H , et al . Edge computation offloading with content caching in 6G-enabled IoV [J ] . IEEE Transactions on Intelligent Transportation Systems , 2024 , 25 ( 3 ): 2733 - 2747 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

边缘智能下基于强化学习的车联网路由协议

面向新能源汽车能源交易系统的PoRT共识机制

基于新型可净化多重签名的车联网高效假名证书分发方案

面向双层区块链的人车分离信任管理方案

车联网联邦学习的数据异质性问题及基于个性化的解决方法综述