Forwarding efficiency aware traffic scheduling algorithm based on deep reinforcement learning

Zongxuan SHA; Ru HUO; Chuang SUN; Shuo WANG; Tao HUANG

doi:10.11959/j.issn.1000-436x.2022148

您当前的位置：

首页 >

文章列表页 >

Forwarding efficiency aware traffic scheduling algorithm based on deep reinforcement learning

Papers | 更新时间：2024-06-05

- Forwarding efficiency aware traffic scheduling algorithm based on deep reinforcement learning
- Journal on Communications Vol. 43, Issue 8, Pages: 30-40(2022)
- 作者机构：
  
  1. 北京工业大学信息学部，北京 100124
  2. 网络通信与安全紫金山实验室, 江苏南京 211111
  3. 清华大学自动化系，北京 100084
  4. 北京邮电大学网络与交换国家重点实验室，北京 100876
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.1000-436x.2022148
  CLC： TN911.5
- Online First：2022-08，
  
  Published：25 August 2022
- 稿件说明：
移动端阅览
Zongxuan SHA, Ru HUO, Chuang SUN, et al. Forwarding efficiency aware traffic scheduling algorithm based on deep reinforcement learning[J]. Journal on Communications, 2022, 43(8): 30-40.
DOI：

Zongxuan SHA, Ru HUO, Chuang SUN, et al. Forwarding efficiency aware traffic scheduling algorithm based on deep reinforcement learning[J]. Journal on Communications, 2022, 43(8): 30-40. DOI： 10.11959/j.issn.1000-436x.2022148.

摘要

软件定义网络（SDN）通过将控制平面与数据平面分离，可实现灵活的流量调度，更有效地利用网络资源。但是，流表项数量、设备负载率及连接主机数量增加等因素的共同作用会导致 SDN 交换机的转发效能降低，进而影响端到端的数据传输时延。为了解决上述问题，提出了基于深度强化学习的转发效能感知流量调度算法。首先，将交换机状态信息统一到感知模型中，通过神经网络建立交换机状态信息和转发效能的映射关系。然后，结合网络状态和流量信息，通过深度强化学习产生流量调度策略。最后，通过由最短路径和负载均衡算法产生的专家样本引导模型训练，不仅使模型学习到专家样本的知识以提升性能，同时提升模型训练效率。实验结果表明，与其他算法相比，所提算法不仅使端到端的平均传输时延降低了15.31%，而且保证了网络整体的负载均衡。

Abstract

The software defined network separates the control plane from the data plane to achieve flexible traffic scheduling

which can use network resources more efficiently.However

with the increase of the number of flow entries

load rate

the number of connected hosts

and other factors

the forwarding efficiency of the SDN switch will be reduced

which will affect the end-to-end transmission delay.To solve the above problems

the forwarding efficiency aware traffic scheduling algorithm based on deep reinforcement learning was proposed.First

the switch state was integrated into the perception model

and the mapping relationship between switch state information and forwarding efficiency was established based on neural network.Then

combined with network state and traffic information

traffic scheduling policy was generated through deep reinforcement learning.Finally

the expert samples generated by the shortest path and load balance algorithms could guide the model training

which enabled the model to learn knowledge from expert samples to improve performance and accelerated the training process.The experimental results show that the proposed algorithm not only reduces the average end-to-end transmission delay by 15.31%

but also ensures the overall load balance of the network

compared with other algorithms.

关键词

Keywords

references

周桐庆 , 蔡志平 , 夏竟 , 等 . 基于软件定义网络的流量工程 [J ] . 软件学报 , 2016 , 27 ( 2 ): 394 - 417 .

ZHOU T Q , CAI Z P , XIA J , et al . Traffic engineering for software defined networks [J ] . Journal of Software , 2016 , 27 ( 2 ): 394 - 417 .

MENDIOLA A , ASTORGA J , JACOB E , et al . A survey on the contributions of software-defined networking to traffic engineering [J ] . IEEE Communications Surveys ＆ Tutorials , 2017 , 19 ( 2 ): 918 - 953 .

杨思锦 , 庄雷 , 宋玉 , 等 . 多模态网络中时间敏感网络模态的智能调度机制 [J ] . 通信学报 , 2022 , 43 ( 5 ): 82 - 91 .

YANG S J , ZHUANG L , SONG Y , et al . Intelligent scheduling mechanism of time-sensitive network modal in polymorphic network [J ] . Journal on Communications , 2022 , 43 ( 5 ): 82 - 91 .

SARASWAT S , AGARWAL V , GUPTA H P , et al . Challenges and solutions in software defined networking:a survey [J ] . Journal of Network and Computer Applications , 2019 , 141 : 23 - 58 .

黄韬 , 刘江 , 汪硕 , 等 . 未来网络技术与发展趋势综述 [J ] . 通信学报 , 2021 , 42 ( 1 ): 130 - 150 .

HUANG T , LIU J , WANG S , et al . Survey of the future network technology and trend [J ] . Journal on Communications , 2021 , 42 ( 1 ): 130 - 150 .

LECUN Y , BENGIO Y , HINTON G . Deep learning [J ] . Nature , 2015 , 521 ( 7553 ): 436 - 444 .

SUTTON R S , BARTO A G . Reinforcement learning:an introduction [M ] . Massachusetts : MIT Press , 1998 .

XIAO Y , LIU J , WU J W , et al . Leveraging deep reinforcement learning for traffic engineering:a survey [J ] . IEEE Communications Surveys ＆ Tutorials , 2021 , 23 ( 4 ): 2064 - 2097 .

XIE J F , YU F R , HUANG T , et al . A survey of machine learning techniques applied to software defined networking (SDN):research issues and challenges [J ] . IEEE Communications Surveys ＆ Tutorials , 2019 , 21 ( 1 ): 393 - 430 .

HARTMAN T , HASSIDIM A , KAPLAN H , et al . How to split a flow? [C ] // Proceedings of IEEE INFOCOM . Piscataway:IEEE Press , 2012 : 828 - 836 .

ZHANG J J , YE M H , GUO Z H , et al . CFR-RL:traffic engineering with reinforcement learning in SDN [J ] . IEEE Journal on Selected Areas in Communications , 2020 , 38 ( 10 ): 2249 - 2259 .

ZHANG J J , GUO Z H , YE M H , et al . SmartEntry:mitigating routing update overhead with reinforcement learning for traffic engineering [C ] // Proceedings of the Workshop on Network Meets AI ＆ ML . Piscataway:IEEE Press , 2020 : 1 - 7 .

HUANG X H , YUAN T T , QIAO G H , et al . Deep reinforcement learning for multimedia traffic control in software defined networking [J ] . IEEE Network , 2018 , 32 ( 6 ): 35 - 41 .

HUANG X H , ZENG M , XIE K . Intelligent traffic control for QoS optimization in hybrid SDNs [J ] . Computer Networks , 2021 , 189 : 107877 .

ZHANG Q Y , WANG X W , LV J H , et al . Intelligent content-aware traffic engineering for SDN:an AI-driven approach [J ] . IEEE Network , 2020 , 34 ( 3 ): 186 - 193 .

XU Z Y , TANG J , MENG J S , et al . Experience-driven networking:a deep reinforcement learning based approach [C ] // Proceedings of IEEE Conference on Computer Communications . Piscataway:IEEE Press , 2018 : 1871 - 1879 .

KATO N , FADLULLAH Z M , MAO B M , et al . The deep learning vision for heterogeneous network traffic control:proposal,challenges,and future perspective [J ] . IEEE Wireless Communications , 2017 , 24 ( 3 ): 146 - 153 .

ZHANG X C , WANG Y L , GENG G G , et al . Delay-optimized multicast tree packing in software-defined networks [J ] . IEEE Transactions on Services Computing , 2021 , PP ( 99 ): 1 .

WU T , ZHOU P , WANG B H , et al . Joint traffic control and multi-channel reassignment for core backbone network in SDN-IoT:a multi-agent deep reinforcement learning approach [J ] . IEEE Transactions on Network Science and Engineering , 2021 , 8 ( 1 ): 231 - 245 .

SAHA N , MISRA S , BERA S . Q-flag:QoS-aware flow-rule aggregation in software-defined IoT networks [J ] . IEEE Internet of Things Journal , 2022 , 9 ( 7 ): 4899 - 4906 .

FILALI A , KOBBANE A , ELMACHKOUR M , et al . SDN controller assignment and load balancing with minimum quota of processing capacity [C ] // Proceedings of IEEE International Conference on Communications (ICC) . Piscataway:IEEE Press , 2018 : 1 - 6 .

SAVAS S S , TORNATORE M , DIKBIYIK F , et al . RASCAR:recovery-aware switch-controller assignment and routing in SDN [J ] . IEEE Transactions on Network and Service Management , 2018 , 15 ( 4 ): 1222 - 1234 .

WANG T , LIU F M , XU H . An efficient online algorithm for dynamic SDN controller assignment in data center networks [J ] . IEEE/ACM Transactions on Networking , 2017 , 25 ( 5 ): 2788 - 2801 .

BERA S , MISRA S , SAHA N . Traffic-aware dynamic controller assignment in SDN [J ] . IEEE Transactions on Communications , 2020 , 68 ( 7 ): 4375 - 4382 .

BOUZIDI E H , OUTTAGARTS A , LANGAR R , et al . Dynamic clustering of software defined network switches and controller placement using deep reinforcement learning [J ] . Computer Networks , 2022 ,207:108852.

LIN N , ZHAO Q , ZHAO L , et al . A novel cost-effective controller placement scheme for software-defined vehicular networks [J ] . IEEE Internet of Things Journal , 2021 , 8 ( 18 ): 14080 - 14093 .

MAITY I , MISRA S , MANDAL C . CORE:prediction-based control plane load reduction in software-defined IoT networks [J ] . IEEE Transactions on Communications , 2021 , 69 ( 3 ): 1835 - 1844 .

Views

1910

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

GAT-based decision mechanism for decentralized joint routing and spectrum access

Goal-oriented hierarchical reinforcement learning-based overlay multicast routing method with SDN integration

Multi-cluster computing power resource scheduling algorithm based on DDPG reinforcement learning

Advances in time-deterministic network technology and application research

Task offloading and resource allocation strategy for vehicular edge computing assisted by intelligent reflecting surfaces

Related Author

Zhou Zibo

Ren Baoquan

Zhong Xudong

Liu Qi

Qin Zhen

Ye Miao

Li Fanyou

Wen Peng

Related Institution

Systems Engineering Institute, Academy of Military Science

Air Force Early Warning Academy

School of Information and Communication, Guilin University of Electronic Technology

Guangxi University Engineering Research Center of Cloud Network Convergence and Data Security, Guilin University of Electronic Technology

School of Artificial Intelligence, China University of Mining and Technology-Beijing

AI问答

⁰