Deep reinforcement learning based algorithm for real-time QoS optimization of software-defined security middle platform

Yuancheng LI; Yongtai QIN

doi:10.11959/j.issn.1000-436x.2023090

您当前的位置：

首页 >

文章列表页 >

Deep reinforcement learning based algorithm for real-time QoS optimization of software-defined security middle platform

Papers | 更新时间：2024-06-06

- Deep reinforcement learning based algorithm for real-time QoS optimization of software-defined security middle platform
- Journal on Communications Vol. 44, Issue 5, Pages: 181-192(2023)
- 作者机构：
  
  华北电力大学控制与计算机工程学院，北京 102206
- 作者简介：
- 基金信息：
  
  The State Grid Jiangxi Information ＆ Telecommunication Company Project(52183520007V)
- DOI：10.11959/j.issn.1000-436x.2023090
  CLC： TP393
- Online First：2023-05，
  
  Published：25 May 2023
- 稿件说明：
移动端阅览
Yuancheng LI, Yongtai QIN. Deep reinforcement learning based algorithm for real-time QoS optimization of software-defined security middle platform[J]. Journal on Communications, 2023, 44(5): 181-192.
DOI：

Yuancheng LI, Yongtai QIN. Deep reinforcement learning based algorithm for real-time QoS optimization of software-defined security middle platform[J]. Journal on Communications, 2023, 44(5): 181-192. DOI： 10.11959/j.issn.1000-436x.2023090.

摘要

针对软件定义安全场景中的服务质量（QoS）实时优化方案因安全防护手段与业务场景不匹配而导致的适用困难和性能下降的问题，提出了基于深度强化学习的软件定义安全中台QoS实时优化算法。首先，将碎片化的安全需求与安全基础设施统一到软件定义安全中台云模型中；然后，通过深度强化学习结合云计算技术提高安全中台的实时匹配和动态适应能力；最后，生成满足QoS目标的安全中台资源实时调度策略。实验结果表明，与现有实时算法相比，所提算法不但保证负载均衡，还提高了 18.7%的作业调度成功率以提高服务质量，降低了34.2%的平均响应时间，具有很好的稳健性，更适用于实时环境。

Abstract

To overcome the problem that the real-time optimization of the quality of service (QoS) in software-defined security scenarios was hindered by the mismatch between security protection measures and business scenarios

which led to difficulties in application and performance degradation.

a novel algorithm based on deep reinforcement learning for optimizing QoS in software defined security middle platforms (SDSmp) in real-time was proposed.Firstly

the fragmented security requirements and infrastructure were integrated into the SDSmp cloud model.Then by leveraging the power of deep reinforcement learning and cloud computing technology

the real-time matching and dynamic adaptation capabilities of the security middle platform were enhanced.Finally

a real-time scheduling strategy for security middle platform resources that meet QoS goals was generated.Experimental results demonstrate that compared to existing real-time methods

the proposed algorithm not only ensures load balancing but also improves job success rate by 18.7% for high QoS and reduces the average response time by 34.2%

and it is highly robust and better suited for real-time environments than existing methods.

关键词

Keywords

references

LIU Y B , LU X Y , JIAN Y , et al . SDSA:a framework of a software-defined security architecture [J ] . China Communications , 2016 , 13 ( 2 ): 178 - 188 .

ALHAJ A N , DUTTA N . Analysis of security attacks in SDN network:a comprehensive survey [C ] // Proceedings of Contemporary Issues in Communication,Cloud and Big Data Analytics . Berlin:Springer , 2022 : 27 - 37 .

QIU R X , QIN Y T , LI Y C , et al . A software-defined security middle platform architecture [C ] // Proceedings of the 5th International Conference on Computer Science and Software Engineering . New York:ACM Press , 2022 : 647 - 651 .

WEI Y , PAN L , LIU S J , et al . DRL-scheduling:an intelligent QoS-aware job scheduling framework for applications in clouds [J ] . IEEE Access , 2018 , 6 : 55112 - 55125 .

ZHANG G , QIU X F , CHANG W . Scheduling of security resources in software defined security architecture [C ] // Proceedings of 2017 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery (CyberC) . Piscataway:IEEE Press , 2018 : 494 - 503 .

MOUSSAID N E , TOUMANARI A , AZHARI M E . Security analysis as software-defined security for SDN environment [C ] // Proceedings of the Fourth International Conference on Software Defined Systems . Piscataway:IEEE Press , 2017 : 87 - 92 .

MOHAMED A , HAMDAN M , KHAN S , et al . Software-defined networks for resource allocation in cloud computing:a survey [J ] . Computer Networks , 2021 ,195:108151.

QI C , WU J X , HU H C , et al . Dynamic-scheduling mechanism of controllers based on security policy in software-defined network [J ] . Electronics Letters , 2016 , 52 ( 23 ): 1918 - 1920 .

LUO S , BEN S M . Orchestration of software-defined security services [C ] // Proceedings of 2016 IEEE International Conference on Communications Workshops . Piscataway:IEEE Press , 2016 : 436 - 441 .

兰巨龙 , 张学帅 , 胡宇翔 , 等 . 基于深度强化学习的软件定义网络QoS优化 [J ] . 通信学报 , 2019 , 40 ( 12 ): 60 - 67 .

LAN J L , ZHANG X S , HU Y X , et al . Software-defined networking QoS optimization based on deep reinforcement learning [J ] . Journal on Communications , 2019 , 40 ( 12 ): 60 - 67 .

吴平 , 常朝稳 , 左志斌 , 等 . 基于地址重载的SDN分组转发验证 [J ] . 通信学报 , 2022 , 43 ( 3 ): 88 - 100 .

WU P , CHANG C W , ZUO Z B , et al . Address overloading-based packet forwarding verification in SDN [J ] . Journal on Communications , 2022 , 43 ( 3 ): 88 - 100 .

MORALES E F , ZARAGOZA J H . An introduction to reinforcement learning [J ] . Machine Learning , 2011 :doi.10.4018/978-1-60960-165-2.Ch004.

MNIH V , KAVUKCUOGLU K , SILVER D , et al . Playing atari with deep reinforcement learning [J ] . arXiv Preprint,arXiv:1312.5602 , 2013 .

MNIH V , KAVUKCUOGLU K , SILVER D , et al . Human-level control through deep reinforcement learning [J ] . Nature , 2015 , 518 ( 7540 ): 529 - 533 .

CHENG L , KALAPGAR A , JAIN A , et al . Cost-aware real-time job scheduling for hybrid cloud using deep reinforcement learning [J ] . Neural Computing and Applications , 2022 , 34 ( 21 ): 18579 - 18593 .

ABUNDO M , DI-VALERIO V , CARDELLINI V , et al . QoS-aware bidding strategies for VM spot instances:a reinforcement learning approach applied to periodic long running jobs [C ] // Proceedings of 2015 IFIP/IEEE International Symposium on Integrated Network Management . Piscataway:IEEE Press , 2015 : 53 - 61 .

ARULKUMARAN K , DEISENROTH M P , BRUNDAGE M , et al . Deep reinforcement learning:a brief survey [J ] . IEEE Signal Processing Magazine , 2017 , 34 ( 6 ): 26 - 38 .

盛妍 , 朱青 , 张明杰 , 等 . 基于数据中台的智能标签关键技术研究与应用 [J ] . 电子技术应用 , 2022 , 48 ( 3 ): 73 - 77 .

SHENG Y , ZHU Q , ZHANG M J , et al . Research and application on key technology of intelligent tag based on data middle platform [J ] . Application of Electronic Technique , 2022 , 48 ( 3 ): 73 - 77 .

CHENG F , HUANG Y F , TANPURE B , et al . Cost-aware job scheduling for cloud instances using deep reinforcement learning [J ] . Cluster Computing , 2022 , 25 ( 1 ): 619 - 631 .

HUANG Y F , CHENG L , XUE L T , et al . Deep adversarial imitation reinforcement learning for QoS-aware cloud job scheduling [J ] . IEEE Systems Journal , 2022 , 16 ( 3 ): 4232 - 4242 .

JAIN V , KUMAR B . QoS-aware task offloading in fog environment using multi-agent deep reinforcement learning [J ] . Journal of Network and Systems Management , 2022 , 31 ( 1 ): 1 - 32 .

CHO C , SHIN S , JEON H , et al . QoS-aware workload distribution in hierarchical edge clouds:a reinforcement learning approach [J ] . IEEE Access , 2020 , 8 : 193297 - 193313 .

LILLICRAP T P , HUNT J J , PRITZEL A , et al . Continuous control with deep reinforcement learning [J ] . arXiv Preprint,arXiv:1509.02971 , 2015 .

NEJAD M M , MASHAYEKHY L , GROSU D . Truthful greedy mechanisms for dynamic virtual machine provisioning and allocation in clouds [J ] . IEEE Transactions on Parallel and Distributed Systems , 2015 , 26 ( 2 ): 594 - 603 .

WANG L , GELENBE E . Adaptive dispatching of tasks in the cloud [J ] . IEEE Transactions on Cloud Computing , 2015 , 6 ( 1 ): 33 - 45 .

Views

586

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Software-defined networking QoS optimization based on deep reinforcement learning

GAT-based decision mechanism for decentralized joint routing and spectrum access

Multi-cluster computing power resource scheduling algorithm based on DDPG reinforcement learning

Research on flexible computing scheduling mechanism for Internet of computing resource

Task offloading and resource allocation strategy for vehicular edge computing assisted by intelligent reflecting surfaces

Related Author

Julong LAN

Xueshuai ZHANG

Yuxiang HU

Penghao SUN

Zhou Zibo

Ren Baoquan

Zhong Xudong

Liu Qi

Related Institution

National Digital Switching System Engineering ＆Research Center

Systems Engineering Institute, Academy of Military Science

Air Force Early Warning Academy

School of Artificial Intelligence, China University of Mining and Technology-Beijing

Computer Network information Center, Chinese Academy of Sciences

AI问答

⁰