Multi-agent caching distribution strategy for content freshness guarantee in IoV

CUI Yaping; SHI Hongji; WU Dapeng; HE Peng; WANG Ruyan

doi:10.11959/j.issn.1000-436x.2025013

您当前的位置：

首页 >

文章列表页 >

Multi-agent caching distribution strategy for content freshness guarantee in IoV

Papers | 更新时间：2025-02-13

- Multi-agent caching distribution strategy for content freshness guarantee in IoV
- Journal on Communications Vol. 46, Issue 1, Pages: 52-66(2025)
- 作者机构：
  
  1.重庆邮电大学通信与信息工程学院，重庆 400065
  2.先进网络与智能互联技术重庆市高校重点实验室，重庆 400065
  3.泛在感知与互联重庆市重点实验室，重庆 400065
- 作者简介：
- 基金信息：
  
  The National Natural Science Foundation of China(61801065;62271096;61871062;U20A20157;62061007);The Science and Technology Research Program of Chongqing Municipal Education Commission(KJQN202000603;KJQN202300621);The Natural Science Foundation of Chongqing(CSTB2022NSCQ-MSX0468;cstc2020jcyjzdxmX0024;cstc2021jcyjmsxmX0892;CSTB2023NSCQ-LZX0134);Sichuan Science and Technology Program(2024YFHZ0093)
- DOI：10.11959/j.issn.1000-436x.2025013
  CLC： TN927
- Received：25 September 2024，
  
  Revised：2024-12-10，
  
  Published：25 January 2025
- 稿件说明：
移动端阅览
崔亚平,石宏吉,吴大鹏等.内容新鲜度保障的车联网多智能体缓存分发策略[J].通信学报,2025,46(01):52-66.

CUI Yaping,SHI Hongji,WU Dapeng,et al.Multi-agent caching distribution strategy for content freshness guarantee in IoV[J].Journal on Communications,2025,46(01):52-66.
崔亚平,石宏吉,吴大鹏等.内容新鲜度保障的车联网多智能体缓存分发策略[J].通信学报,2025,46(01):52-66. DOI： 10.11959/j.issn.1000-436x.2025013.

CUI Yaping,SHI Hongji,WU Dapeng,et al.Multi-agent caching distribution strategy for content freshness guarantee in IoV[J].Journal on Communications,2025,46(01):52-66. DOI： 10.11959/j.issn.1000-436x.2025013.

摘要

车辆需要频繁动态变化内容支持车联网（IoV）时延敏感型应用，这会增加宏基站（MBS）负载，降低内容新鲜度。利用边缘缓存将最新内容提前缓存在小基站（SBS）能有效降低车辆时延和提高内容新鲜度。对影响时延和内容信息年龄（AoI）进行深入分析，提出一种内容新鲜度保障的多智能体强化学习（MARL）算法，通过优化缓存分发决策保障车辆获得高新鲜度内容。仿真结果表明，所提算法不仅收敛速度更快，而且在降低车辆时延和提升内容新鲜度方面表现出更好效果。

Abstract

Vehicles need to dynamically changing content to support latency-sensitive applications in Internet of vehicles (IoV)

thereby increasing the load on the macro base station (MBS) and reducing the freshness of content. Utilizing edge caching to cache the latest content in small base station (SBS) can effectively reduce the latency and improve the content freshness. An in-depth analysis was conducted on latency and content's age of information (AoI). A content freshness assurance multi-agent reinforcement learning (MARL) algorithm was proposed

which optimized cache distribution decisions to guarantee high freshness. Simulation results show that the proposed algorithm not only converges faster but also demonstrates better performance in reducing latency and enhancing content freshness.

关键词

Keywords

references

NAN Z J , JIA Y J , REN Z , et al . Delay-aware content delivery with deep reinforcement learning in Internet of vehicles [J ] . IEEE Transactions on Intelligent Transportation Systems , 2022 , 23 ( 7 ): 8918 - 8929 .

ZHANG K , LENG S P , HE Y J , et al . Mobile edge computing and networking for green and low-latency Internet of things [J ] . IEEE Communications Magazine , 2018 , 56 ( 5 ): 39 - 45 .

LI Y , XIA S C , ZHENG M Y , et al . Lyapunov optimization-based trade-off policy for mobile cloud offloading in heterogeneous wireless networks [J ] . IEEE Transactions on Cloud Computing , 2022 , 10 ( 1 ): 491 - 505 .

LI Y , LIU J , CAO B , et al . Joint optimization of radio and virtual machine resources with uncertain user demands in mobile cloud computing [J ] . IEEE Transactions on Multimedia , 2018 , 20 ( 9 ): 2427 - 2438 .

JIANG K , SUN C , ZHOU H , et al . Intelligence-empowered mobile edge computing: framework, issues, implementation, and outlook [J ] . IEEE Network , 2021 , 35 ( 5 ): 74 - 82 .

NING Z L , ZHANG K Y , WANG X J , et al . Intelligent edge computing in Internet of vehicles: a joint computation offloading and caching solution [J ] . IEEE Transactions on Intelligent Transportation Systems , 2021 , 22 ( 4 ): 2212 - 2225 .

ZHOU J W , CHEN F F , HE Q , et al . Data caching optimization with fairness in mobile edge computing [J ] . IEEE Transactions on Services Computing , 2023 , 16 ( 3 ): 1750 - 1762 .

WANG R Y , KAN Z W , CUI Y P , et al . Cooperative caching strategy with content request prediction in Internet of vehicles [J ] . IEEE Internet of Things Journal , 2021 , 8 ( 11 ): 8964 - 8975 .

WU H H , FAN Y Z , JIN J C , et al . Social-aware decentralized cooperative caching for Internet of vehicles [J ] . IEEE Internet of Things Journal , 2023 , 10 ( 16 ): 14834 - 14845 .

QIAO G H , LENG S P , MAHARJAN S , et al . Deep reinforcement learning for cooperative content caching in vehicular edge computing and networks [J ] . IEEE Internet of Things Journal , 2020 , 7 ( 1 ): 247 - 257 .

TRAN T X , POMPILI D . Adaptive bitrate video caching and processing in mobile-edge computing networks [J ] . IEEE Transactions on Mobile Computing , 2019 , 18 ( 9 ): 1965 - 1978 .

ZHANG T K , FANG X Y , LIU Y W , et al . D2D-enabled mobile user edge caching: a multi-winner auction approach [J ] . IEEE Transactions on Vehicular Technology , 2019 , 68 ( 12 ): 12314 - 12328 .

HUANG X G , XU K , CHEN Q B , et al . Delay-aware caching in Internet-of-vehicles networks [J ] . IEEE Internet of Things Journal , 2021 , 8 ( 13 ): 10911 - 10921 .

YANG P , LYU F , WU W , et al . Edge coordinated query configuration for low-latency and accurate video analytics [J ] . IEEE Transactions on Industrial Informatics , 2020 , 16 ( 7 ): 4855 - 4864 .

ZHANG M , ARAFA A , HUANG J W , et al . Pricing fresh data [J ] . IEEE Journal on Selected Areas in Communications , 2021 , 39 ( 5 ): 1211 - 1225 .

TIAN H , XU X L , QI L Y , et al . CoPace: edge computation offloading and caching for self-driving with deep reinforcement learning [J ] . IEEE Transactions on Vehicular Technology , 2021 , 70 ( 12 ): 13281 - 13293 .

YATES R D , SUN Y , BROWN D R , et al . Age of information: an introduction and survey [J ] . IEEE Journal on Selected Areas in Communications , 2021 , 39 ( 5 ): 1183 - 1210 .

ZHANG S , WANG L D , LUO H B , et al . Age of information and delay tradeoff with freshness-aware mobile edge cache update [C ] // Proceedings of the 2019 IEEE Global Communications Conference (GLOBECOM) . Piscataway : IEEE Press , 2019 : 1 - 6 .

AHANI G , YUAN D . Optimal content caching and recommendation with age of information [J ] . IEEE Transactions on Mobile Computing , 2024 , 23 ( 1 ): 689 - 704 .

ZHANG S , LI J J , LUO H B , et al . Towards fresh and low-latency content delivery in vehicular networks: an edge caching aspect [C ] // Proceedings of the 2018 10th International Conference on Wireless Communications and Signal Processing (WCSP) . Piscataway : IEEE Press , 2018 : 1 - 6 .

ZHANG S , LI J J , LUO H B , et al . Low-latency and fresh content provision in information-centric vehicular networks [J ] . IEEE Transactions on Mobile Computing , 2022 , 21 ( 5 ): 1723 - 1738 .

AHANI G , YUAN D . Accounting for information freshness in scheduling of content caching [C ] // Proceedings of the ICC 2020 - 2020 IEEE International Conference on Communications (ICC) . Piscataway : IEEE Press , 2020 : 1 - 6 .

王红艳 , 孙其博 , 马骁 , 等 . 边缘辅助实时应用中信息年龄感知的任务调度 [J ] . 通信学报 , 2024 , 45 ( 6 ): 144 - 159 .

WANG H Y , SUN Q B , MA X , et al . AoI-aware task scheduling in edge-assisted real-time applications [J ] . Journal on Communications , 2024 , 45 ( 6 ): 144 - 159 .

张政 , 谢鑫 , 柏桐 , 等 . 信息年龄和能耗联合优化的无线体域网卸载和调度策略 [J ] . 通信学报 , 2024 , 45 ( 9 ): 92 - 100 .

ZHANG Z , XIE X , BAI T , et al . Joint optimization strategy of age of information and energy consumption for offloading and scheduling in WBAN [J ] . Journal on Communications , 2024 , 45 ( 9 ): 92 - 100 .

ZHANG D G , WANG W J , ZHANG J , et al . Novel edge caching approach based on multi-agent deep reinforcement learning for Internet of vehicles [J ] . IEEE Transactions on Intelligent Transportation Systems , 2023 , 24 ( 8 ): 8324 - 8338 .

MA M Y , WONG V W S . Age of information driven cache content update scheduling for dynamic contents in heterogeneous networks [J ] . IEEE Transactions on Wireless Communications , 2020 , 19 ( 12 ): 8427 - 8441 .

XU C , WANG X J , YANG H H , et al . AoI and energy consumption oriented dynamic status updating in caching enabled IoT networks [C ] // Proceedings of the IEEE INFOCOM 2020-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS) . Piscataway : IEEE Press , 2020 : 710 - 715 .

WU X W , LI X H , LI J , et al . Caching transient content for IoT sensing: multi-agent soft actor-critic [J ] . IEEE Transactions on Communications , 2021 , 69 ( 9 ): 5886 - 5901 .

LIANG L , YE H , LI G Y . Spectrum sharing in vehicular networks based on multi-agent reinforcement learning [J ] . IEEE Journal on Selected Areas in Communications , 2019 , 37 ( 10 ): 2282 - 2292 .

LIU Y , YU H M , XIE S L , et al . Deep reinforcement learning for offloading and resource allocation in vehicle edge computing and networks [J ] . IEEE Transactions on Vehicular Technology , 2019 , 68 ( 11 ): 11158 - 11168 .

ZHANG S , WANG L D , LUO H B , et al . AoI-delay tradeoff in mobile edge caching with freshness-aware content refreshing [J ] . IEEE Transactions on Wireless Communications , 2021 , 20 ( 8 ): 5329 - 5342 .

CHU T S , WANG J , CODECÀ L , et al . Multi-agent deep reinforcement learning for large-scale traffic signal control [J ] . IEEE Transactions on Intelligent Transportation Systems , 2020 , 21 ( 3 ): 1086 - 1095 .

IQBAL S , SHA F . Actor-attention-critic for multi-agent reinforcement learning [C ] // Proceedings of the 36th International Conference on Machine Learning . Saarland : DBLP , 2020 : 2961 - 2970 .

ZHAO Y W , LI R B , WANG C Y , et al . Neighboring-aware caching in heterogeneous edge networks by actor-attention-critic learning [C ] // Proceedings of the ICC 2021-IEEE International Conference on Communications . Piscataway : IEEE Press , 2021 : 1 - 6 .

CHEN S Z , HU J L , SHI Y , et al . Vehicle-to-everything (V2X) services supported by LTE-based systems and 5G [J ] . IEEE Communications Standards Magazine , 2017 , 1 ( 2 ): 70 - 76 .

HE P , CAO L , CUI Y P , et al . Multi-agent caching strategy for spatial-temporal popularity in IoV [J ] . IEEE Transactions on Vehicular Technology , 2023 , 72 ( 10 ): 13536 - 13546 .

ZHI Y , TIAN J , LIU Q Q , et al . Multi-agent reinforcement learning for cooperative edge caching in heterogeneous networks [C ] // Proceedings of the 2021 13th International Conference on Wireless Communications and Signal Processing (WCSP) . Piscataway : IEEE Press , 2021 : 1 - 6 .

ZHOU X H , BILAL M , DOU R H , et al . Edge computation offloading with content caching in 6G-enabled IoV [J ] . IEEE Transactions on Intelligent Transportation Systems , 2024 , 25 ( 3 ): 2733 - 2747 .

Views

774

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Edge intelligence-assisted routing protocol for Internet of vehicles via reinforcement learning

Cooperative optimization method for connected vehicles and traffic signals via C-V2X

Homomorphic encryption based privacy protection protocol for point-of-interest queries in Internet of vehicles

Cloud-edge computing based dual access control scheme for IoV data

Lightweight authentication key agreement protocol for Internet of vehicles

Related Author

Bingyi LIU

Yuhao LIU

Weizhen HAN

Zhenchang XIA

Libing WU

Shengwu XIONG

Liu Kai

Mei Shiyuan

Related Institution

School of Computer Science and Artificial Intelligence, Wuhan University of Technology

Sanya Science and Education Innovation Park, Wuhan University of Technology

Chongqing Research Institute, Wuhan University of Technology

School of Cyber Science and Engineering, Wuhan University

National Elite Institute of Engineering, Chongqing University

AI问答

⁰