基于深度强化学习的软件定义安全中台QoS实时优化算法

李元诚; 秦永泰

doi:10.11959/j.issn.1000-436x.2023090

您当前的位置：

首页 >

文章列表页 >

基于深度强化学习的软件定义安全中台QoS实时优化算法

学术论文 | 更新时间：2024-06-06

- 基于深度强化学习的软件定义安全中台QoS实时优化算法
- Deep reinforcement learning based algorithm for real-time QoS optimization of software-defined security middle platform
- 通信学报 2023年44卷第5期页码：181-192
- 作者机构：
  
  华北电力大学控制与计算机工程学院，北京 102206
- 作者简介：
  
  [ "李元诚（1970- ），男，山东烟台人，博士，华北电力大学教授、博士生导师，主要研究方向为密码学、信息安全等" ]
  [ "秦永泰（1998- ），男，甘肃定西人，华北电力大学硕士生，主要研究方向为深度强化学习、软件定义安全等" ]
- 基金信息：
  
  国网江西信息通信公司基金资助项目(52183520007V)
- DOI：10.11959/j.issn.1000-436x.2023090
  中图分类号： TP393
- 网络出版日期：2023-05，
  
  纸质出版日期：2023-05-25
- 稿件说明：
移动端阅览
李元诚, 秦永泰. 基于深度强化学习的软件定义安全中台QoS实时优化算法[J]. 通信学报, 2023,44(5):181-192.

Yuancheng LI, Yongtai QIN. Deep reinforcement learning based algorithm for real-time QoS optimization of software-defined security middle platform[J]. Journal on communications, 2023, 44(5): 181-192.
李元诚, 秦永泰. 基于深度强化学习的软件定义安全中台QoS实时优化算法[J]. 通信学报, 2023,44(5):181-192. DOI： 10.11959/j.issn.1000-436x.2023090.

Yuancheng LI, Yongtai QIN. Deep reinforcement learning based algorithm for real-time QoS optimization of software-defined security middle platform[J]. Journal on communications, 2023, 44(5): 181-192. DOI： 10.11959/j.issn.1000-436x.2023090.

摘要

针对软件定义安全场景中的服务质量（QoS）实时优化方案因安全防护手段与业务场景不匹配而导致的适用困难和性能下降的问题，提出了基于深度强化学习的软件定义安全中台QoS实时优化算法。首先，将碎片化的安全需求与安全基础设施统一到软件定义安全中台云模型中；然后，通过深度强化学习结合云计算技术提高安全中台的实时匹配和动态适应能力；最后，生成满足QoS目标的安全中台资源实时调度策略。实验结果表明，与现有实时算法相比，所提算法不但保证负载均衡，还提高了 18.7%的作业调度成功率以提高服务质量，降低了34.2%的平均响应时间，具有很好的稳健性，更适用于实时环境。

Abstract

To overcome the problem that the real-time optimization of the quality of service (QoS) in software-defined security scenarios was hindered by the mismatch between security protection measures and business scenarios

which led to difficulties in application and performance degradation.

a novel algorithm based on deep reinforcement learning for optimizing QoS in software defined security middle platforms (SDSmp) in real-time was proposed.Firstly

the fragmented security requirements and infrastructure were integrated into the SDSmp cloud model.Then by leveraging the power of deep reinforcement learning and cloud computing technology

the real-time matching and dynamic adaptation capabilities of the security middle platform were enhanced.Finally

a real-time scheduling strategy for security middle platform resources that meet QoS goals was generated.Experimental results demonstrate that compared to existing real-time methods

the proposed algorithm not only ensures load balancing but also improves job success rate by 18.7% for high QoS and reduces the average response time by 34.2%

and it is highly robust and better suited for real-time environments than existing methods.

关键词

Keywords

references

LIU Y B , LU X Y , JIAN Y , et al . SDSA:a framework of a software-defined security architecture [J ] . China Communications , 2016 , 13 ( 2 ): 178 - 188 .

ALHAJ A N , DUTTA N . Analysis of security attacks in SDN network:a comprehensive survey [C ] // Proceedings of Contemporary Issues in Communication,Cloud and Big Data Analytics . Berlin:Springer , 2022 : 27 - 37 .

QIU R X , QIN Y T , LI Y C , et al . A software-defined security middle platform architecture [C ] // Proceedings of the 5th International Conference on Computer Science and Software Engineering . New York:ACM Press , 2022 : 647 - 651 .

WEI Y , PAN L , LIU S J , et al . DRL-scheduling:an intelligent QoS-aware job scheduling framework for applications in clouds [J ] . IEEE Access , 2018 , 6 : 55112 - 55125 .

ZHANG G , QIU X F , CHANG W . Scheduling of security resources in software defined security architecture [C ] // Proceedings of 2017 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery (CyberC) . Piscataway:IEEE Press , 2018 : 494 - 503 .

MOUSSAID N E , TOUMANARI A , AZHARI M E . Security analysis as software-defined security for SDN environment [C ] // Proceedings of the Fourth International Conference on Software Defined Systems . Piscataway:IEEE Press , 2017 : 87 - 92 .

MOHAMED A , HAMDAN M , KHAN S , et al . Software-defined networks for resource allocation in cloud computing:a survey [J ] . Computer Networks , 2021 ,195:108151.

QI C , WU J X , HU H C , et al . Dynamic-scheduling mechanism of controllers based on security policy in software-defined network [J ] . Electronics Letters , 2016 , 52 ( 23 ): 1918 - 1920 .

LUO S , BEN S M . Orchestration of software-defined security services [C ] // Proceedings of 2016 IEEE International Conference on Communications Workshops . Piscataway:IEEE Press , 2016 : 436 - 441 .

兰巨龙 , 张学帅 , 胡宇翔 , 等 . 基于深度强化学习的软件定义网络QoS优化 [J ] . 通信学报 , 2019 , 40 ( 12 ): 60 - 67 .

LAN J L , ZHANG X S , HU Y X , et al . Software-defined networking QoS optimization based on deep reinforcement learning [J ] . Journal on Communications , 2019 , 40 ( 12 ): 60 - 67 .

吴平 , 常朝稳 , 左志斌 , 等 . 基于地址重载的SDN分组转发验证 [J ] . 通信学报 , 2022 , 43 ( 3 ): 88 - 100 .

WU P , CHANG C W , ZUO Z B , et al . Address overloading-based packet forwarding verification in SDN [J ] . Journal on Communications , 2022 , 43 ( 3 ): 88 - 100 .

MORALES E F , ZARAGOZA J H . An introduction to reinforcement learning [J ] . Machine Learning , 2011 :doi.10.4018/978-1-60960-165-2.Ch004.

MNIH V , KAVUKCUOGLU K , SILVER D , et al . Playing atari with deep reinforcement learning [J ] . arXiv Preprint,arXiv:1312.5602 , 2013 .

MNIH V , KAVUKCUOGLU K , SILVER D , et al . Human-level control through deep reinforcement learning [J ] . Nature , 2015 , 518 ( 7540 ): 529 - 533 .

CHENG L , KALAPGAR A , JAIN A , et al . Cost-aware real-time job scheduling for hybrid cloud using deep reinforcement learning [J ] . Neural Computing and Applications , 2022 , 34 ( 21 ): 18579 - 18593 .

ABUNDO M , DI-VALERIO V , CARDELLINI V , et al . QoS-aware bidding strategies for VM spot instances:a reinforcement learning approach applied to periodic long running jobs [C ] // Proceedings of 2015 IFIP/IEEE International Symposium on Integrated Network Management . Piscataway:IEEE Press , 2015 : 53 - 61 .

ARULKUMARAN K , DEISENROTH M P , BRUNDAGE M , et al . Deep reinforcement learning:a brief survey [J ] . IEEE Signal Processing Magazine , 2017 , 34 ( 6 ): 26 - 38 .

盛妍 , 朱青 , 张明杰 , 等 . 基于数据中台的智能标签关键技术研究与应用 [J ] . 电子技术应用 , 2022 , 48 ( 3 ): 73 - 77 .

SHENG Y , ZHU Q , ZHANG M J , et al . Research and application on key technology of intelligent tag based on data middle platform [J ] . Application of Electronic Technique , 2022 , 48 ( 3 ): 73 - 77 .

CHENG F , HUANG Y F , TANPURE B , et al . Cost-aware job scheduling for cloud instances using deep reinforcement learning [J ] . Cluster Computing , 2022 , 25 ( 1 ): 619 - 631 .

HUANG Y F , CHENG L , XUE L T , et al . Deep adversarial imitation reinforcement learning for QoS-aware cloud job scheduling [J ] . IEEE Systems Journal , 2022 , 16 ( 3 ): 4232 - 4242 .

JAIN V , KUMAR B . QoS-aware task offloading in fog environment using multi-agent deep reinforcement learning [J ] . Journal of Network and Systems Management , 2022 , 31 ( 1 ): 1 - 32 .

CHO C , SHIN S , JEON H , et al . QoS-aware workload distribution in hierarchical edge clouds:a reinforcement learning approach [J ] . IEEE Access , 2020 , 8 : 193297 - 193313 .

LILLICRAP T P , HUNT J J , PRITZEL A , et al . Continuous control with deep reinforcement learning [J ] . arXiv Preprint,arXiv:1509.02971 , 2015 .

NEJAD M M , MASHAYEKHY L , GROSU D . Truthful greedy mechanisms for dynamic virtual machine provisioning and allocation in clouds [J ] . IEEE Transactions on Parallel and Distributed Systems , 2015 , 26 ( 2 ): 594 - 603 .

WANG L , GELENBE E . Adaptive dispatching of tasks in the cloud [J ] . IEEE Transactions on Cloud Computing , 2015 , 6 ( 1 ): 33 - 45 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于深度强化学习的软件定义网络QoS优化

基于多智能体深度强化学习的低轨星座跳波束资源调度研究

基于图卷积神经网络的超密集物联网资源分配策略

基于样本信息熵辅助的深度强化学习抗干扰策略

毫米波车联网多基站多用户下的安全传输方案