Software-defined networking QoS optimization based on deep reinforcement learning

Julong LAN; Xueshuai ZHANG; Yuxiang HU; Penghao SUN

doi:10.11959/j.issn.1000-436x.2019227

您当前的位置：

首页 >

文章列表页 >

Software-defined networking QoS optimization based on deep reinforcement learning

Papers | 更新时间：2024-06-05

- Software-defined networking QoS optimization based on deep reinforcement learning
- Journal on Communications Vol. 40, Issue 12, Pages: 60-67(2019)
- 作者机构：
  
  国家数字交换系统工程技术研究中心，河南郑州 450001
- 作者简介：
- 基金信息：
  
  The National Network Security Special Program of China(2017YFB0803204);The Innovation Group Program Project of National Natural Science Foundation of China(61521003);The National Natural Science Foundation of China(61702547);The National Natural Science Foundation of China(61872382);The Research and Development Program in Key Areas of Guangdong Province(2018B010113001)
- DOI：10.11959/j.issn.1000-436x.2019227
  CLC： TP393
- Online First：2019-12，
  
  Published：25 December 2019
- 稿件说明：
移动端阅览
Julong LAN, Xueshuai ZHANG, Yuxiang HU, et al. Software-defined networking QoS optimization based on deep reinforcement learning[J]. Journal on Communications, 2019, 40(12): 60-67.
DOI：

Julong LAN, Xueshuai ZHANG, Yuxiang HU, et al. Software-defined networking QoS optimization based on deep reinforcement learning[J]. Journal on Communications, 2019, 40(12): 60-67. DOI： 10.11959/j.issn.1000-436x.2019227.

摘要

为解决软件定义网络场景中，当前主流的基于启发式算法的QoS优化方案常因参数与网络场景不匹配出现性能下降的问题，提出了基于深度强化学习的软件定义网络QoS优化算法。首先将网络资源和状态信息统一到网络模型中，然后通过长短期记忆网络提升算法的流量感知能力，最后基于深度强化学习生成满足QoS目标的动态流量调度策略。实验结果表明，相对于现有算法，所提算法不但保证了端到端传输时延和分组丢失率，而且提高了22.7%的网络负载均衡程度，增加了8.2%的网络吞吐率。

Abstract

To solve the problem that the QoS optimization schemes which based on heuristic algorithm degraded often due to the mismatch between parameters and network characteristics in software-defined networking scenarios

a software-defined networking QoS optimization algorithm based on deep reinforcement learning was proposed.Firstly

the network resources and state information were integrated into the network model

and then the flow perception capability was improved by the long short-term memory

and finally the dynamic flow scheduling strategy

which satisfied the specific QoS objectives

were generated in combination with deep reinforcement learning.The experimental results show that

compared with the existing algorithms

the proposed algorithm not only ensures the end-to-end delay and packet loss rate

but also improves the network load balancing by 22.7% and increases the throughput by 8.2%.

关键词

Keywords

references

MCKEOWN N . Software-defined networking [J ] . INFOCOM Keynote Talk , 2009 , 17 ( 2 ): 30 - 32 .

张朝昆 , 崔勇 , 唐翯祎 , 等 . 软件定义网络（SDN）研究进展 [J ] . 软件学报 , 2015 , 26 ( 1 ): 62 - 81 .

ZHANG C K , CUI Y , TANG H Y , et al . State-of-the-art survey on software-defined-networking (SDN) [J ] . Journal of Software , 2015 , 26 ( 1 ): 62 - 81 .

董谦 , 李俊 , 马宇翔 , 等 . 软件定义网络中基于分段路由的流量调度方法 [J ] . 通信学报 , 2018 , 39 ( 11 ): 23 - 35 .

DONG Q , LI J , MA Y X , et al . Traffic scheduling method based on segment routing in software-defined networking [J ] . Journal on Com-munications , 2018 , 39 ( 11 ): 23 - 35 .

HARTMAN T , HASSIDIM A , KAPLAN H , et al . How to split a flow? [C ] // 2012 Proceedings IEEE INFOCOM . IEEE , 2012 : 828 - 836 .

ZHANG S R , . Valiant load-balancing:building networks that can support all traffic matrices [M ] // London:Algorithms for Next Generation Networks . Springer , 2010 : 19 - 30 .

SUTTON R S , BARTO A G . Reinforcement learning:an introduction [M ] . MIT Press , 2018 .

LECUN Y , BENGIO Y , HINTON G . Deep learning [J ] . Nature , 2015 , 521 ( 7553 ):436.

MAO H , NETRAVALI R , ALIZADEH M . Neural adaptive video streaming with pensieve [C ] // The Conference of the ACM Special Interest Group on Data Communication . ACM , 2017 : 197 - 210 .

XIAO S , HE D , GONG Z . Deep-q:traffic-driven QoS inference using deep generative network [C ] // The 2018 Workshop on Network Meets AI ＆ ML . ACM , 2018 : 67 - 73 .

CHEN L , CHEN K , BAI W , et al . Scheduling mix-flows in commodity datacenters with karuna [C ] // The 2016 ACM SIGCOMM Conference . ACM , 2016 : 174 - 187 .

ONGARO F , CERQUEIRA E , FOSCHINI L , et al . Enhancing the quality level support for real-time multimedia applications in software-defined networks [C ] // 2015 International Conference on Computing,Networking and Communications (ICNC) . IEEE , 2015 : 505 - 509 .

ALIZADEH M , EDSALL T , DHARMAPURIKAR S , et al . CONGA:distributed congestion-aware load balancing for datacenters [J ] . ACM SIGCOMM Computer Communication Review , 2014 , 44 ( 4 ): 503 - 514 .

XU Z , TANG J , MENG J , et al . Experience-driven networking:a deep reinforcement learning based approach [C ] // IEEE INFOCOM 2018 IEEE Conference on Computer Communications . IEEE , 2018 : 1871 - 1879 .

XU Z , TANG J , YIN C , et al . Experience-driven congestion control:when multi-path TCP meets deep reinforcement learning [J ] . IEEE Journal on Selected Areas in Communications , 2019 , 37 ( 6 ): 1325 - 1336 .

MCKEOWN N , ANDERSON T , BALAKRISHNAN H , et al . OpenFlow:enabling innovation in campus networks [J ] . ACM SIGCOMM Computer Communication Review , 2008 , 38 ( 2 ): 69 - 74 .

BOSSHART P , DALY D , GIBB G , et al . P4:programming protocol-independent packet processors [J ] . ACM SIGCOMM Computer Communication Review , 2014 , 44 ( 3 ): 87 - 95 .

ZHOU W , LI L , LUO M , et al . REST API design patterns for SDN northbound API [C ] // 2014 28th International Conference on Advanced Information Networking and Applications Workshops . IEEE , 2014 : 358 - 365 .

CLEMM A , CHANDRAMOULI M , KRISHNAMURTHY S . DNA:an SDN framework for distributed network analytics [C ] // 2015 IFIP/IEEE International Symposium on Integrated Network Management (IM) . IEEE , 2015 : 9 - 17 .

LILLICRAP T P , HUNT J J , PRITZEL A , et al . Continuous control with deep reinforcement learning [C ] // Eighth International Conference on Learning Representations . ICLR , 2016 : 187 - 200 .

RÄCKE H , . Optimal hierarchical decompositions for congestion minimization in networks [C ] // The Fortieth Annual ACM Symposium on Theory of Computing . ACM , 2008 : 255 - 264 .

Views

2274

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Deep reinforcement learning based algorithm for real-time QoS optimization of software-defined security middle platform

GAT-based decision mechanism for decentralized joint routing and spectrum access

Multi-cluster computing power resource scheduling algorithm based on DDPG reinforcement learning

Research on flexible computing scheduling mechanism for Internet of computing resource

Task offloading and resource allocation strategy for vehicular edge computing assisted by intelligent reflecting surfaces

Related Author

Yuancheng LI

Yongtai QIN

Zhou Zibo

Ren Baoquan

Zhong Xudong

Liu Qi

Qin Zhen

HU Yahui

Related Institution

School of Control and Computer Engineering, North China Electric Power University

Systems Engineering Institute, Academy of Military Science

Air Force Early Warning Academy

School of Artificial Intelligence, China University of Mining and Technology-Beijing

Computer Network information Center, Chinese Academy of Sciences

AI问答

⁰