Cooperative service caching and peer offloading in Internet of vehicles based on multi-agent meta-reinforcement learning

Zhaolong NING; Kaiyuan ZHANG; Xiaojie WANG; Lei GUO

doi:10.11959/j.issn.1000-436x.2021104

您当前的位置：

首页 >

文章列表页 >

Cooperative service caching and peer offloading in Internet of vehicles based on multi-agent meta-reinforcement learning

Papers | 更新时间：2024-06-05

- Cooperative service caching and peer offloading in Internet of vehicles based on multi-agent meta-reinforcement learning
- Journal on Communications Vol. 42, Issue 6, Pages: 118-130(2021)
- 作者机构：
  
  1. 重庆邮电大学通信与信息工程学院，重庆 400065
  2. 大连理工大学软件学院，辽宁大连 116620
- 作者简介：
- 基金信息：
  
  The National Key Research and Development Program of China(2018YFE0206800);The National Natural Science Foundation of China(62025105);The National Natural Science Foundation of China(61971084);The National Natural Science Foundation of China(62001073);Chongqing Talent Program(CQYC2020058659)
- DOI：10.11959/j.issn.1000-436x.2021104
  CLC： TN92
- Online First：2021-06，
  
  Published：25 June 2021
- 稿件说明：
移动端阅览
Zhaolong NING, Kaiyuan ZHANG, Xiaojie WANG, et al. Cooperative service caching and peer offloading in Internet of vehicles based on multi-agent meta-reinforcement learning[J]. Journal on Communications, 2021, 42(6): 118-130.
DOI：

Zhaolong NING, Kaiyuan ZHANG, Xiaojie WANG, et al. Cooperative service caching and peer offloading in Internet of vehicles based on multi-agent meta-reinforcement learning[J]. Journal on Communications, 2021, 42(6): 118-130. DOI： 10.11959/j.issn.1000-436x.2021104.

摘要

为了降低求解优化问题的难度，提出一种双层的多路侧单元（RSU）协同缓存框架将问题进行解耦。外层采用多智能体元强化学习方法，在每个本地智能体进行决策学习的同时，采用长短期记忆网络作为元智能体来平衡本地决策并加速学习过程，从而得到最优的 RSU 缓存策略；内层采用拉格朗日乘子法求解最佳协同卸载策略，实现 RSU 间的任务分配。基于杭州真实交通数据的实验表明，所提方法具有理想的能效性能，并且能够在大规模任务流下保持网络稳健性。

Abstract

In order to reduce computation complexity

a two-layer mutli-RSU (road side unit) service caching and peer offloading algorithm (MPO) was proposed to decouple the optimization problem.In the designed MPO

the outer layer utilized multi-agent meta-reinforcement learning

which established long short-term memory (LSTM) network as the meta-agent to balance decisions of local agents and accelerate learning progress.The inner layer utilized lagrange multiplier method to achieve optimal decision for RSU peer offloading.Experimental results based on real traffic data in Hangzhou demonstrate that the proposed method outperforms other methods and remains robust under large-scale workloads.

关键词

Keywords

references

NING Z L , DONG P R , WANG X J , et al . Mobile edge computing enabled 5G health monitoring for Internet of medical things:a decentralized game theoretic approach [J ] . IEEE Journal on Selected Areas in Communications , 2021 , 39 ( 2 ): 463 - 478 .

谢人超 , 廉晓飞 , 贾庆民 , 等 . 移动边缘计算卸载技术综述 [J ] . 通信学报 , 2018 , 39 ( 11 ): 138 - 155 .

XIE R C , LIAN X F , JIA Q M , et al . Survey on computation offloading in mobile edge computing [J ] . Journal on Communications , 2018 , 39 ( 11 ): 138 - 155 .

ZHUANG W H , YE Q , LYU F , et al . SDN/NFV-empowered future IoV with enhanced communication,computing,and caching [J ] . Proceedings of the IEEE , 2020 , 108 ( 2 ): 274 - 291 .

WANG X J , NING Z L , GUO S . Multi-agent imitation learning for pervasive edge computing:a decentralized computation offloading algorithm [J ] . IEEE Transactions on Parallel and Distributed Systems , 2021 , 32 ( 2 ): 411 - 425 .

刘雷 , 陈晨 , 冯杰 , 等 . 车载边缘计算中任务卸载和服务缓存的联合智能优化 [J ] . 通信学报 , 2021 , 42 ( 1 ): 18 - 26 .

LIU L , CHEN C , FENG J , et al . Joint intelligent optimization of task offloading and service caching for vehicular edge computing [J ] . Journal on Communications , 2021 , 42 ( 1 ): 18 - 26 .

CHEN Z Q , DUAN L Y , WANG S Q , et al . Toward knowledge as a service over networks:a deep learning model communication paradigm [J ] . IEEE Journal on Selected Areas in Communications , 2019 , 37 ( 6 ): 1349 - 1363 .

张彦 , 张科 , 曹佳钰 . 边缘智能驱动的车联网 [J ] . 物联网学报 , 2018 , 2 ( 4 ): 40 - 48 .

ZHANG Y , ZHANG K , CAO J Y . Internet of vehicles empowered by edge intelligence [J ] . Chinese Journal on Internet of Things , 2018 , 2 ( 4 ): 40 - 48 .

WANG X J , NING Z L , GUO S , et al . Imitation learning enabled task scheduling for online vehicular edge computing [J ] . IEEE Transactions on Mobile Computing , 2020 , PP ( 99 ): 1 .

NING Z L , DONG P R , WANG X J , et al . When deep reinforcement learning meets 5G-enabled vehicular networks:a distributed offloading framework for traffic big data [J ] . IEEE Transactions on Industrial Informatics , 2020 , 16 ( 2 ): 1352 - 1361 .

NING Z L , DONG P R , WANG X J , et al . Deep reinforcement learning for vehicular edge computing:an intelligent offloading system [J ] . ACM Transactions on Intelligent Systems and Technology , 2019 , 10 ( 6 ): 60 .

TRAN T X , HAJISAMI A , PANDEY P , et al . Collaborative mobile edge computing in 5G networks:new paradigms,scenarios,and challenges [J ] . IEEE Communications Magazine , 2017 , 55 ( 4 ): 54 - 61 .

亓晋 , 孙海蓉 , 巩锟 , 等 . 移动边缘计算中基于信誉值的智能计算卸载模型研究 [J ] . 通信学报 , 2020 , 41 ( 7 ): 141 - 151 .

QI J , SUN H R , GONG K , et al . Research on intelligent computing offloading model based on reputation value in mobile edge computing [J ] . Journal on Communications , 2020 , 41 ( 7 ): 141 - 151 .

XU J , CHEN L X , ZHOU P . Joint service caching and task offloading for mobile edge computing in dense networks [C ] // IEEE INFOCOM 2018 - IEEE Conference on Computer Communications . Piscataway:IEEE Press , 2018 : 207 - 215 .

WANG X J , NING Z L , WANG L . Offloading in Internet of vehicles:a fog-enabled real-time traffic management system [J ] . IEEE Transactions on Industrial Informatics , 2018 , 14 ( 10 ): 4568 - 4578 .

CHEN M , HAO Y X . Task offloading for mobile edge computing in software defined ultra-dense network [J ] . IEEE Journal on Selected Areas in Communications , 2018 , 36 ( 3 ): 587 - 597 .

TAN L T , HU R Q , HANZO L . Twin-timescale artificial intelligence aided mobility-aware edge caching and computing in vehicular networks [J ] . IEEE Transactions on Vehicular Technology , 2019 , 68 ( 4 ): 3086 - 3099 .

DAI Y Y , XU D , MAHARJAN S , et al . Joint computation offloading and user association in multi-task mobile edge computing [J ] . IEEE Transactions on Vehicular Technology , 2018 , 67 ( 12 ): 12313 - 12325 .

TAO O Y , ZHI Z , XU C . Follow me at the edge:mobility-aware dynamic service placement for mobile edge computing [C ] // 2018 IEEE/ACM 26th International Symposium on Quality of Service . Piscataway:IEEE Press , 2018 : 1 - 10 .

NING Z L , DONG P R , KONG X J , et al . A cooperative partial computation offloading scheme for mobile edge computing enabled Internet of Things [J ] . IEEE Internet of Things Journal , 2019 , 6 ( 3 ): 4804 - 4814 .

QIAO G H , LENG S P , ZHANG K , et al . Collaborative task offloading in vehicular edge multi-access networks [J ] . IEEE Communications Magazine , 2018 , 56 ( 8 ): 48 - 54 .

NEWELL G F . Simple queueing systems [M ] . Dordrecht : Springer Netherlands , 1982 .

HERINGS J J , PEETERS R . Stationary equilibria in stochastic games:structure,selection,and computation [J ] . Research Memorandum , 2000 , 118 ( 1 ): 32 - 60 .

Views

1585

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Multi-indicator intention-driven adaptive optimization strategy for UAV computation offloading and trajectory planning

Cooperative optimization method for connected vehicles and traffic signals via C-V2X

Homomorphic encryption based privacy protection protocol for point-of-interest queries in Internet of vehicles

Cloud-edge computing based dual access control scheme for IoV data

Lightweight authentication key agreement protocol for Internet of vehicles

Related Author

Lin Peng

Huang Xinliang

Ning Zhaolong

Liu Yan

Guo Lei

Zhang Zhizhong

Liu Kai

Mei Shiyuan

Related Institution

School of Computer Science and Engineering, Northeastern University

School of Electronic and Information Engineering, Nanjing University of Information Science and Technology

Key Laboratory of Intelligent Support Technology for Complex Environments, Ministry of Education, Nanjing University of Information Science and Technology

Shanghai Yinwang Intelligent Technology Co., Ltd.

College of Computer Science, Chongqing University

AI问答

⁰