抗拜占庭攻击的梯度净化联邦自适应学习算法

杨辉; 邱子游; 李中美; 朱建勇

doi:10.11959/j.issn.1000-436x.2024209

您当前的位置：

首页 >

文章列表页 >

抗拜占庭攻击的梯度净化联邦自适应学习算法

学术论文 | 更新时间：2025-01-08

- 抗拜占庭攻击的梯度净化联邦自适应学习算法
- Gradient purification federated adaptive learning algorithm for Byzantine attack resistance
- 通信学报 2024年45卷第Z1期页码：1-11
- 作者机构：
  
  1.华东交通大学电气与自动化工程学院，江西南昌 330013
  2.华东理工大学信息科学与工程学院，上海 200237
- 作者简介：
  
  [ "杨辉（1965- ），男，江西高安人，华东交通大学教授、博士生导师，主要研究方向为复杂工业过程建模与优化控制、轨道交通自动化与运行优化。" ]
  [ "邱子游（1998- ），男，广东佛山人，华东交通大学硕士生，主要研究方向为端边云协同控制平台、数字孪生。" ]
  [ "李中美（1989- ），女，江苏苏州人，博士，华东理工大学硕士生导师，主要研究方向为人工智能（感知、认知、决策）、工业生产过程建模与优化控制。" ]
  [ "朱建勇（1977- ），男，江西新干人，博士，华东交通大学教授，主要研究方向为复杂工业过程控制与优化、大数据分析。" ]
- 基金信息：
  
  国家自然科学基金资助项目(62363010;61733005);工业控制技术国家重点实验室开放课题基金资助项目(ICT2024B50)
- DOI：10.11959/j.issn.1000-436x.2024209
  中图分类号： TN92
- 收稿日期：2024-08-16，
  
  纸质出版日期：2024-10-25
- 稿件说明：
移动端阅览
杨辉,邱子游,李中美等.抗拜占庭攻击的梯度净化联邦自适应学习算法[J].通信学报,2024,45(Z1):1-11.

YANG Hui,QIU Ziyou,LI Zhongmei,et al.Gradient purification federated adaptive learning algorithm for Byzantine attack resistance[J].Journal on Communications,2024,45(Z1):1-11.
杨辉,邱子游,李中美等.抗拜占庭攻击的梯度净化联邦自适应学习算法[J].通信学报,2024,45(Z1):1-11. DOI： 10.11959/j.issn.1000-436x.2024209.

YANG Hui,QIU Ziyou,LI Zhongmei,et al.Gradient purification federated adaptive learning algorithm for Byzantine attack resistance[J].Journal on Communications,2024,45(Z1):1-11. DOI： 10.11959/j.issn.1000-436x.2024209.

摘要

在工业大数据之下，数据安全和隐私保护是关键挑战之一。传统的数据共享和模型训练方法在应对数据泄露和恶意攻击（尤其是复杂的拜占庭攻击和投毒攻击）时效果有限，因为传统联邦学习通常假定所有参与方都是可信的，这使得模型在遭遇投毒攻击时性能显著下降。为解决这个问题，本文提出一种抗拜占庭攻击的梯度净化联邦自适应学习算法，通过滑动窗口梯度过滤器和符号聚类过滤器识别恶意梯度，滑动窗口方法检测异常梯度，而符号聚类则根据梯度方向一致性筛选出偏离的对抗性梯度，经过过滤后，使用基于权重的自适应聚合规则对剩余的可信梯度进行加权聚合，动态调整参与方梯度的权重，降低恶意梯度的影响，从而增强模型的鲁棒性。实验结果显示，尽管新型投毒攻击的强度更高，但所提算法能有效防御这些攻击且减轻模型性能的损失。相比于传统防御算法，所提算法不仅提高了模型的准确性，还提升了其安全性。

Abstract

In the context of industrial big data

data security and privacy are key challenges. Traditional data-sharing and model-training methods struggle against risks like Byzantine and poisoning attacks

as federated learning typically assumes all participants are trustworthy

leading to performance drops under attacks. To address this

a Byzantine-resilient gradient purification federated adaptive learning algorithm was proposed. The malicious gradients were identified through a sliding window gradient filter and a sign-based clustering filter. The sliding window method detected anomalous gradients

while the sign-based clustering filter selected adversarial gradients based on the consistency of gradient directions. After filtering

a weight-based adaptive aggregation rule was applied to perform weighted aggregation on the remaining trustworthy gradients

dynamically adjusting the weights of participant gradients to reduce the impact of malicious gradients

thereby enhancing the model’s robustness. Experimental results show that despite the increased intensity of new poisoning attacks

the proposed algorithm effectively defends against these attacks while minimizing the loss in model performance. Compared to traditional defense algorithms

it not only improves model accuracy but also enhances its security.

关键词

Keywords

references

微众银行AI项目组 . 联邦学习白皮书 V1.0 [R ] . 2018 .

WeBank AI Project Team . Federated learning white paper V1.0 [R ] . 2018 .

潘碧莹 , 丘海华 , 张家伦 . 不同数据分布的联邦机器学习技术研究 [C ] // 5G网络创新研讨会(2019)论文集 . 广州 : 移动通信 , 2019 : 271 - 276 .

PAN B Y , QING H H , ZHANG J L . Research on federal machine learning technology with different data distribution [C ] // 5G Network Innovation Seminar (2019) Proceedings . Guangzhou : Mobile Communications , 2019 : 271 - 276 .

YANG Q , LIU Y , CHEN T J , et al . Federated machine learning [J ] . ACM Transactions on Intelligent Systems and Technology , 2019 , 10 ( 2 ): 1 - 19 .

WANG S Q , TUOR T , SALONIDIS T , et al . Adaptive federated learning in resource constrained edge computing systems [J ] . IEEE Journal on Selected Areas in Communications , 2019 , 37 ( 6 ): 1205 - 1221 .

WANG X , LIU X , ZHAO J . Federated adaptive learning: a new approach to optimizing federated learning systems [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2022 , 33 ( 5 ): 1234 - 1247 .

YANG Z , LIU W , ZHANG W . Byzantine-robust federated learning: a review [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2021 , 32 ( 5 ): 1722 - 1735 .

XIE C , KOYEJO O , GUPTA I . Zeno: distributed stochastic gradient descent with suspicion-based fault-tolerance [J ] . arXiv Preprint , arXiv: 1805.10032 , 2018 .

WEN J , ZHANG Z X , LAN Y , et al . A survey on federated learning: challenges and applications [J ] . International Journal of Machine Learning and Cybernetics , 2023 , 14 ( 2 ): 513 - 535 .

ENTHOVEN D , AL-ARS Z . An overview of federated deep learning privacy attacks and defensive strategies [C ] // Studies in Computational Intelligence . Berlin : Springer , 2021 : 173 - 196 .

YIN D , CHEN Y , RAMCHANDRAN K , et al . Byzantine-robust distributed learning: towards optimal statistical rates [J ] . arXiv Preprint , arXiv: 1803.01498 , 2018 .

KRUM F , ALISTARH D , ANGELOVA M . Bayesian inference for identifying and isolating malicious clients in federated learning [C ] // Proceedings of the 2021 ACM Conference on Computer and Communications Security . New York : ACM Press , 2021 : 1121 - 1135 .

ZHANG J , WANG T , WANG S . Privacy-preserving federated learning via homomorphic encryption and secure multi-party computation [C ] // Proceedings of the 2022 Network and Distributed System Security Symposium (NDSS) . Piscataway : IEEE Press , 2022 : 95 - 110 .

LI X , HUANG K , LIU Y . A reweighting aggregation rule for mitigating malicious updates in federated learning [C ] // Proceedings of the 29th USENIX Security Symposium . Berkeley : USENIX Association , 2020 : 789 - 803 .

刘晶 , 张喆语 , 董志红 , 等 . 基于工业物联网的区块链多目标优化 [J ] . 计算机集成制造系统 , 2021 , 27 ( 8 ): 2382 - 2392 .

LIU J , ZHANG Z Y , DONG Z H , et al . Multi-objective optimization of blockchain-based on industrial Internet of things [J ] . Computer Integrated Manufacturing Systems , 2021 , 27 ( 8 ): 2382 - 2392 .

ZHUANG W M , WEN Y G , ZHANG X S , et al . Performance optimization of federated person re-identification via benchmark analysis [C ] // Proceedings of the 28th ACM International Conference on Multimedia . New York : ACM Press , 2020 : 955 - 963 .

SHELLER M J , EDWARDS B , REINA G A , et al . Federated learning in medicine: facilitating multi-institutional collaborations without sharing patient data [J ] . Scientific Reports , 2020 , 10 ( 1 ): 12598 .

WeBank . Utilization of FATE in risk management of credit in small and micro enterprises [R ] . 2019 .

PREUVENEERS D , RIMMER V , TSINGENOPOULOS I , et al . Chained anomaly detection models for federated learning: an intrusion detection case study [J ] . Applied Sciences , 2018 , 8 ( 12 ): 2663 .

ZHU X D , LI H , YU Y . Blockchain-based privacy-preserving deep learning [C ] // Proceedings of the International Conference on Information Security and Cryptology . Berlin : Springer , 2019 : 370 - 383 .

KIM Y J , HONG C S . Blockchain-based node-aware dynamic weighting methods for improving federated learning performance [C ] // Proceedings of the 2019 20th Asia-Pacific Network Operations and Management Symposium (APNOMS) . Piscataway : IEEE Press , 2019 : 1 - 4 .

SUN J W , LI A , WANG B H , et al . Soteria: provable defense against privacy leakage in federated learning from representation perspective [C ] // Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE Press , 2021 : 9307 - 9315 .

BARUCH M , BARUCH G , GOLDBERG Y . A little is enough: circumventing defenses for distributed learning [C ] // Proceedings of the 33rd International Conference on Neural Information Processing Systems . New York : ACM Press , 2019 : 8635 - 8645 .

XIE C , KOYEJO S , GUPTA I . Zeno++: robust fully asynchronous SGD [J ] . arXiv Preprint , arXiv: 1903.07020 , 2019 .

FANG M H , CAO X Y , JIA J Y , et al . Local model poisoning attacks to Byzantine-robust federated learning [C ] // 29th USENIX Security Symposium (USENIX Security 20) . Berkeley : USENIX Association , 2020 : 1605 - 1622 .

BIGGIO B , NELSON B , LASKOV P . Poisoning attacks against support vector machines [C ] // Proceedings of the 29th International Conference on Machine Learning . Piscataway : IEEE Press , 2012 : 1467 - 1474 .

KEOGH E , LIN J . Clustering of time-series subsequences is meaningless: implications for previous and future research [J ] . Knowledge and Information Systems , 2005 , 8 ( 2 ): 154 - 177 .

YIN D , CHEN Y D , RAMCHANDRAN K , et al . Byzantine-robust distributed learning: towards optimal statistical rates [C ] // Proceedings of the 35th International Conference on Machine Learning . Piscataway : IEEE Press , 2018 : 5650 - 5659 .

COMANICIU D , MEER P . Mean shift: a robust approach toward feature space analysis [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2002 , 24 ( 5 ): 603 - 619 .

BLANCHARD P , MHAMDI E M E , GUERRAOUI R , et al . Byzantine-tolerant machine learning [C ] // Proceedings of the 31st International Conference on Neural Information Processing Systems . New York : ACM Press , 2017 : 118 - 128 .

LI T , SAHU A K , ZAHEER M , et al . Federated learning: challenges, methods, and future directions [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2021 , 32 ( 9 ): 3787 - 3807 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

联邦学习中的拜占庭攻防研究综述

基于自适应拜占庭防御的安全联邦学习方案

面向Non-IID数据的拜占庭鲁棒联邦学习

空天地一体化网络中基于联邦深度强化学习的边缘协作缓存策略

基于零集中差分隐私的联邦学习激励方案