基于深度强化学习的应急物联网切片资源预留算法

孙国林; 欧睿杰; 刘贵松

doi:10.11959/j.issn.1000-436x.2020200

您当前的位置：

首页 >

文章列表页 >

基于深度强化学习的应急物联网切片资源预留算法

专题：面向智慧应急的通信与计算融合 | 更新时间：2024-06-05

- 基于深度强化学习的应急物联网切片资源预留算法
- Deep reinforcement learning-based resource reservation algorithm for emergency Internet-of-things slice
- 通信学报 2020年41卷第9期页码：8-20
- 作者机构：
  
  1. 电子科技大学计算机科学与工程学院，四川成都 611731
  2. 电子科技大学中山学院，广东中山 528402
- 作者简介：
  
  [ "孙国林（1978- ），男，河北唐山人，博士，电子科技大学副教授、硕士生导师，主要研究方向为人工智能、区块链和移动智能系统等" ]
  [ "欧睿杰（1989- ），男，四川成都人，电子科技大学博士生，主要研究方向为人工智能、区块链、移动网络资源管理等" ]
  [ "刘贵松（1973- ），男，山东临沂人，博士，电子科技大学教授、博士生导师，主要研究方向为类脑计算、机器学习和模式识别与智能系统等" ]
- 基金信息：
  
  国家自然科学基金资助项目(61771098);四川省科技计划基金资助项目(2020YFQ0025)
- DOI：10.11959/j.issn.1000-436x.2020200
  中图分类号： TN92
- 网络出版日期：2020-09，
  
  纸质出版日期：2020-09-25
- 稿件说明：
移动端阅览
孙国林, 欧睿杰, 刘贵松. 基于深度强化学习的应急物联网切片资源预留算法[J]. 通信学报, 2020,41(9):8-20.

Guolin SUN, Ruijie OU, Guisong LIU. Deep reinforcement learning-based resource reservation algorithm for emergency Internet-of-things slice[J]. Journal on communications, 2020, 41(9): 8-20.
孙国林, 欧睿杰, 刘贵松. 基于深度强化学习的应急物联网切片资源预留算法[J]. 通信学报, 2020,41(9):8-20. DOI： 10.11959/j.issn.1000-436x.2020200.

Guolin SUN, Ruijie OU, Guisong LIU. Deep reinforcement learning-based resource reservation algorithm for emergency Internet-of-things slice[J]. Journal on communications, 2020, 41(9): 8-20. DOI： 10.11959/j.issn.1000-436x.2020200.

摘要

针对应急物联网（EIoT）超低时延服务需求，设计了面向超低时延传输应急物联网的多切片网络架构，提出 EIoT 切片资源预留和多异构切片资源共享与隔离的方法框架。所提框架采用深度强化学习方法实现实时异构切片间资源需求的自动预测与分配，切片内用户资源分配建模为基于形状的二维背包问题并采用启发式算法数值求解，从而实现切片内资源定制化。仿真结果表明，基于资源预留的方法能够使 EIoT 切片显式保留资源，提供了更好的安全隔离级别；深度强化学习能够保证资源预留的准确和实时更新，有效兼顾资源利用率和切片差异化服务质量要求。与4个已有算法对比表明，Dueling DQN具有更好的性能优势。

Abstract

Based on the requirements of ultra-low latency services for emergency Internet-of-things (EIoT) applications

a multi-slice network architecture for ultra-low latency emergency IoT was designed

and a general methodology framework based on resource reservation

sharing and isolation for multiple slices was proposed.In the proposed framework

real-time and automatic inter-slice resource demand prediction and allocation were realized based on deep reinforcement learning (DRL)

while intra-slice user resource allocation was modeled as a shape-based 2-dimension packing problem and solved with a heuristic numerical algorithm

so that intra-slice resource customization was achieved.Simulation results show that the resource reservation-based method enable EIoT slices to explicitly reserve resources

provide a better security isolation level

and DRL could guarantee accuracy and real-time updates of resource reservations.Compared with four existing algorithms

dueling deep Q-network (DQN) performes better than the benchmarks.

关键词

Keywords

references

FOUKAS X . Network slicing in 5G:survey and challenges [J ] . IEEE Communications Magazine , 2017 , 55 ( 5 ): 80 - 87 .

KALOXYLOS A . A survey and an analysis of network slicing in 5G networks [J ] . IEEE Communications Standards Magazine , 2018 , 2 ( 1 ): 60 - 65 .

YOUSAF F Z , SCIANCALEPORE V , LIEBSCH M , et al . MANOaaS:a multi-tenant NFV MANO for 5G network slices [J ] . IEEE Communications Magazine , 2019 , 57 ( 5 ): 103 - 109 .

COSTA-PEREZ X , SWETINA J , GUO T , et al . Radio access network virtualization for future mobile carrier networks [J ] . IEEE Communications Magazine , 2013 , 51 ( 7 ): 27 - 35 .

ZAKI Y , ZHAO L , GOERG C , et al . LTE wireless virtualization and spectrum management [C ] // Wireless ＆ Mobile Networking Conference . Piscataway:IEEE Press , 2010 : 1 - 6 .

BHANAGE G , SESKAR I , MAHINDRA R , et al . Virtual base station:architecture for an open shared WiMAX framework [C ] // Second ACM SIGCOMM Workshop on Virtualized Infrastructure Systems and Architectures . New York:ACM Press , 2010 : 1 - 10 .

KOKKU R , MAHINDRA R , ZHANG H , et al . NVS:a substrate for virtualizing wireless resources in cellular networks [J ] . IEEE/ACM Transactions on Networking , 2012 , 20 ( 5 ): 1333 - 1346 .

KOKKU R , MAHINDRA R , ZHANG H , et al . Cell-Slice:cellular wireless resource slicing for active RAN sharing [C ] // Fifth International Conference on Communication Systems and Networks . Piscataway:IEEE Press , 2013 : 1 - 10 .

PETERS S W , HEATH R W . The future of WiMAX:multi-hop relaying with IEEE 802.16j [J ] . IEEE Communications Magazine , 2009 , 47 ( 1 ): 104 - 111 .

HOLMA H , TOSKALA A . LTE for UMTS-OFDMA and SC-FDMA based radio access [M ] . New Jersey : John Wiley ＆ SonsPress , 2009 .

MAHINDRA R , KHOJASTEPOUR M A , ZHANG H , et al . Radio access network sharing in cellular networks [C ] // 21st IEEE International Conference on Network Protocols . Piscataway:IEEE Press , 2013 : 1 - 10 .

LIANG C , YU F R . Distributed resource allocation in virtualized wireless cellular networks based on ADMM [C ] // 2015 IEEE Conference on Computer Communications Workshops . Piscataway:IEEE Press , 2015 : 360 - 365 .

LEE Y L , LOO J , CHUAH T C , et al . Dynamic network slicing for multitenant heterogeneous cloud radio access networks [J ] . IEEE Transactions on Wireless Communications , 2018 , 17 ( 4 ): 2146 - 2161 .

AIJAZ A . Hap-SliceR:a radio resource slicing framework for 5G networks with haptic communications [J ] . IEEE Systems Journal , 2018 , 12 ( 3 ): 2285 - 2296 .

XIANG H , YAN S , ANDPENG M . A realization of fog-RAN slicing via deep reinforcement learning [J ] . IEEE Transactions on Wireless Communications , 2020 , 19 ( 4 ): 2515 - 2527 .

SUN G , GEBREKIDAN Z T , BOATENG G O , et al . Dynamic reservation and deep reinforcement learning based autonomous resource slicing for virtualized radio access networks [J ] . IEEE Access , 2019 , 7 ( 1 ): 45758 - 45772 .

ROSS S . Introduction to probability models,11 th ed [M ] . Salt Lake City : Academic PressPress , 2014 .

TANG L , ZHANG Y , LIANG R , et al . Virtual resource allocation algorithm for network utility maximization based on network slicing [J ] . Journal of Electronics ＆ Information Technology , 2017 , 39 ( 8 ): 1812 - 1818 .

刘全 , 翟建伟 , 章宗长 , 等 . 深度强化学习综述 [J ] . 计算机学报 , 2017 , 40 ( 1 ): 1 - 27 .

LIU Q , ZHAI J W , ZHANG Z Z , et al . A survey on deep reinforcement learning [J ] . Chinese Journal of Computers , 2018 , 41 ( 1 ): 1 - 27 .

LIU F , REN X , LIU Y , et al . simNet:stepwise image-topic merging network for generating detailed and comprehensive image captions [C ] // Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing . Stroudsburg:ACL Press , 2018 : 137 - 149 .

GONZALEZ A , KUEHLMORGEN S , FESTAG A , et al . Resource allocation for block-based multi-carrier systems considering QoS requirements [C ] // 2017 IEEE Global Communications Conference . Piscataway:IEEE Press , 2017 : 1 - 7 .

RICHART M , BALIOSIAN J , SERRAT J , et al . Resource slicing in virtual wireless networks:a survey [J ] . IEEE Transactions on Network and Service Management , 2016 , 13 ( 3 ): 1 - 15 .

马康 , 高尚 . 分布估计算法求解矩形件排样优化问题 [J ] . 电子设计工程 , 2017 , 25 ( 2 ): 49 - 54 .

MA K , GAO S . Solution to optimize cutting pattern in rectangular packing problem based on estimation of distribution algorithm [J ] . Electronic Design Engineering , 2017 , 25 ( 2 ): 49 - 54

曾兆敏 , 王继红 , 管卫利 . 二维板材切割下料问题的一种确定性算法 [J ] . 图学学报 , 2016 , 37 ( 4 ): 471 - 475 .

ZENG Z M , WANG J H , GUAN W L . A deterministic algorithm for solving the problem of two-dimensional sheet cutting stock [J ] . Journal of Graphics , 2016 , 37 ( 4 ): 471 - 475 .

CHAZELLE B . The bottom-left bin-packing heuristic:an efficient implementation [J ] . IEEE Transactions on Computers , 1983 , C-32 ( 8 ): 697 - 707 .

HELMERSSON K W , ANSARI J . Ultra-reliable and low-latency communication for wireless factory automation:From LTE to 5G [C ] // IEEE 21st International Conference on Emerging Technologies and Factory Automation . Piscataway:IEEE Press , 2016 : 1 - 8 .

MAMMAN M , HANAPI Z H , ABDULLAH A , et al . Quality of service class identifier (QCI) radio resource allocation algorithm for LTE downlink [J ] . PLoS One , 2019 , 14 ( 1 ):e0210310.

浏览量

1237

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于多智能体深度强化学习的低轨星座跳波束资源调度研究

基于图卷积神经网络的超密集物联网资源分配策略

基于样本信息熵辅助的深度强化学习抗干扰策略

毫米波车联网多基站多用户下的安全传输方案

基于深度强化学习的空天地一体化网络信息物理系统垂直切换策略