基于深度强化学习的软件定义网络QoS优化

兰巨龙; 张学帅; 胡宇翔; 孙鹏浩

doi:10.11959/j.issn.1000-436x.2019227

您当前的位置：

首页 >

文章列表页 >

基于深度强化学习的软件定义网络QoS优化

学术论文 | 更新时间：2024-06-05

- 基于深度强化学习的软件定义网络QoS优化
- Software-defined networking QoS optimization based on deep reinforcement learning
- 通信学报 2019年40卷第12期页码：60-67
- 作者机构：
  
  国家数字交换系统工程技术研究中心，河南郑州 450001
- 作者简介：
  
  [ "兰巨龙（1962- ），男，河北张家口人，博士，国家数字交换系统工程技术研究中心教授、博士生导师，主要研究方向为未来信息通信网络关键理论与技术" ]
  [ "张学帅（1994- ），男，山东菏泽人，国家数字交换系统工程技术研究中心硕士生，主要研究方向为软件定义网络" ]
  [ "胡宇翔（1982- ），男，河南周口人，博士，国家数字交换系统工程技术研究中心副教授、博士生导师，主要研究方向为未来网络关键技术、网络智慧化等" ]
  [ "孙鹏浩（1992- ），男，山东青岛人，国家数字交换系统工程技术研究中心博士生，主要研究方向为软件定义网络、流量工程等" ]
- 基金信息：
  
  国家网络空间安全专项基金资助项目(2017YFB0803204);国家自然科学基金创新群体基金资助项目(61521003);国家自然科学基金资助项目(61702547);国家自然科学基金资助项目(61872382);广东省重点领域研发计划基金资助项目(2018B010113001)
- DOI：10.11959/j.issn.1000-436x.2019227
  中图分类号： TP393
- 网络首发：2019-12，
  
  纸质出版：2019-12-25
- 稿件说明：
移动端阅览
兰巨龙, 张学帅, 胡宇翔, 等. 基于深度强化学习的软件定义网络QoS优化[J]. 通信学报, 2019,40(12):60-67.

Julong LAN, Xueshuai ZHANG, Yuxiang HU, et al. Software-defined networking QoS optimization based on deep reinforcement learning[J]. Journal on Communications, 2019, 40(12): 60-67.
兰巨龙, 张学帅, 胡宇翔, 等. 基于深度强化学习的软件定义网络QoS优化[J]. 通信学报, 2019,40(12):60-67. DOI： 10.11959/j.issn.1000-436x.2019227.

Julong LAN, Xueshuai ZHANG, Yuxiang HU, et al. Software-defined networking QoS optimization based on deep reinforcement learning[J]. Journal on Communications, 2019, 40(12): 60-67. DOI： 10.11959/j.issn.1000-436x.2019227.

摘要

为解决软件定义网络场景中，当前主流的基于启发式算法的QoS优化方案常因参数与网络场景不匹配出现性能下降的问题，提出了基于深度强化学习的软件定义网络QoS优化算法。首先将网络资源和状态信息统一到网络模型中，然后通过长短期记忆网络提升算法的流量感知能力，最后基于深度强化学习生成满足QoS目标的动态流量调度策略。实验结果表明，相对于现有算法，所提算法不但保证了端到端传输时延和分组丢失率，而且提高了22.7%的网络负载均衡程度，增加了8.2%的网络吞吐率。

Abstract

To solve the problem that the QoS optimization schemes which based on heuristic algorithm degraded often due to the mismatch between parameters and network characteristics in software-defined networking scenarios

a software-defined networking QoS optimization algorithm based on deep reinforcement learning was proposed.Firstly

the network resources and state information were integrated into the network model

and then the flow perception capability was improved by the long short-term memory

and finally the dynamic flow scheduling strategy

which satisfied the specific QoS objectives

were generated in combination with deep reinforcement learning.The experimental results show that

compared with the existing algorithms

the proposed algorithm not only ensures the end-to-end delay and packet loss rate

but also improves the network load balancing by 22.7% and increases the throughput by 8.2%.

关键词

Keywords

references

MCKEOWN N . Software-defined networking [J ] . INFOCOM Keynote Talk , 2009 , 17 ( 2 ): 30 - 32 .

张朝昆 , 崔勇 , 唐翯祎 , 等 . 软件定义网络（SDN）研究进展 [J ] . 软件学报 , 2015 , 26 ( 1 ): 62 - 81 .

ZHANG C K , CUI Y , TANG H Y , et al . State-of-the-art survey on software-defined-networking (SDN) [J ] . Journal of Software , 2015 , 26 ( 1 ): 62 - 81 .

董谦 , 李俊 , 马宇翔 , 等 . 软件定义网络中基于分段路由的流量调度方法 [J ] . 通信学报 , 2018 , 39 ( 11 ): 23 - 35 .

DONG Q , LI J , MA Y X , et al . Traffic scheduling method based on segment routing in software-defined networking [J ] . Journal on Com-munications , 2018 , 39 ( 11 ): 23 - 35 .

HARTMAN T , HASSIDIM A , KAPLAN H , et al . How to split a flow? [C ] // 2012 Proceedings IEEE INFOCOM . IEEE , 2012 : 828 - 836 .

ZHANG S R , . Valiant load-balancing:building networks that can support all traffic matrices [M ] // London:Algorithms for Next Generation Networks . Springer , 2010 : 19 - 30 .

SUTTON R S , BARTO A G . Reinforcement learning:an introduction [M ] . MIT Press , 2018 .

LECUN Y , BENGIO Y , HINTON G . Deep learning [J ] . Nature , 2015 , 521 ( 7553 ):436.

MAO H , NETRAVALI R , ALIZADEH M . Neural adaptive video streaming with pensieve [C ] // The Conference of the ACM Special Interest Group on Data Communication . ACM , 2017 : 197 - 210 .

XIAO S , HE D , GONG Z . Deep-q:traffic-driven QoS inference using deep generative network [C ] // The 2018 Workshop on Network Meets AI ＆ ML . ACM , 2018 : 67 - 73 .

CHEN L , CHEN K , BAI W , et al . Scheduling mix-flows in commodity datacenters with karuna [C ] // The 2016 ACM SIGCOMM Conference . ACM , 2016 : 174 - 187 .

ONGARO F , CERQUEIRA E , FOSCHINI L , et al . Enhancing the quality level support for real-time multimedia applications in software-defined networks [C ] // 2015 International Conference on Computing,Networking and Communications (ICNC) . IEEE , 2015 : 505 - 509 .

ALIZADEH M , EDSALL T , DHARMAPURIKAR S , et al . CONGA:distributed congestion-aware load balancing for datacenters [J ] . ACM SIGCOMM Computer Communication Review , 2014 , 44 ( 4 ): 503 - 514 .

XU Z , TANG J , MENG J , et al . Experience-driven networking:a deep reinforcement learning based approach [C ] // IEEE INFOCOM 2018 IEEE Conference on Computer Communications . IEEE , 2018 : 1871 - 1879 .

XU Z , TANG J , YIN C , et al . Experience-driven congestion control:when multi-path TCP meets deep reinforcement learning [J ] . IEEE Journal on Selected Areas in Communications , 2019 , 37 ( 6 ): 1325 - 1336 .

MCKEOWN N , ANDERSON T , BALAKRISHNAN H , et al . OpenFlow:enabling innovation in campus networks [J ] . ACM SIGCOMM Computer Communication Review , 2008 , 38 ( 2 ): 69 - 74 .

BOSSHART P , DALY D , GIBB G , et al . P4:programming protocol-independent packet processors [J ] . ACM SIGCOMM Computer Communication Review , 2014 , 44 ( 3 ): 87 - 95 .

ZHOU W , LI L , LUO M , et al . REST API design patterns for SDN northbound API [C ] // 2014 28th International Conference on Advanced Information Networking and Applications Workshops . IEEE , 2014 : 358 - 365 .

CLEMM A , CHANDRAMOULI M , KRISHNAMURTHY S . DNA:an SDN framework for distributed network analytics [C ] // 2015 IFIP/IEEE International Symposium on Integrated Network Management (IM) . IEEE , 2015 : 9 - 17 .

LILLICRAP T P , HUNT J J , PRITZEL A , et al . Continuous control with deep reinforcement learning [C ] // Eighth International Conference on Learning Representations . ICLR , 2016 : 187 - 200 .

RÄCKE H , . Optimal hierarchical decompositions for congestion minimization in networks [C ] // The Fortieth Annual ACM Symposium on Theory of Computing . ACM , 2008 : 255 - 264 .

浏览量

2166

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于深度强化学习的软件定义安全中台QoS实时优化算法

基于DDPG强化学习的多集群算力资源调度算法

面向算力互联网的柔性计算调度机制研究

智能反射表面辅助的车载边缘任务卸载和资源分配策略

数字孪生辅助下基于D3QN的车载网络协同卸载算法