基于特征投影与自适应增强的联邦蒸馏方法

陈宁江; 章德华

doi:10.11959/j.issn.1000-436x.TXXB260116

您当前的位置：

首页 >

文章列表页 >

基于特征投影与自适应增强的联邦蒸馏方法

更新时间：2026-05-14

- 基于特征投影与自适应增强的联邦蒸馏方法
- A Federated Distillation Method Based on Feature Projection and Adaptive Enhancement
- 通信学报 2026年
- 作者机构：
  
  1.广西大学计算机与电子信息学院，广西南宁 530004
  2.广西智能数字服务技术创新中心，广西南宁 530004
  3.广西高校并行分布与智能计算重点实验室，广西南宁 530004
  4.广西人工智能学院，广西南宁 530004
- 作者简介：
  
  章德华，2313394042@st.gxu.edu.cn
- 基金信息：
  
  国家自然科学基金资助项目;The National Natural Science Foundation of China(62162003);广西重点研发计划项目(GuikeAB25069258;GuikeAB25069130)
- DOI：10.11959/j.issn.1000-436x.TXXB260116
  中图分类号： TP309
- 收稿：2026-03-09，
  
  修回：2026-05-13，
  
  录用：2026-05-14，
- 稿件说明：
移动端阅览
陈宁江, 章德华. 基于特征投影与自适应增强的联邦蒸馏方法[J/OL]. 通信学报, 2026.

Chen Ningjiang, Zhang Dehua. A Federated Distillation Method Based on Feature Projection and Adaptive Enhancement[J/OL]. Journal on Communications, 2026.
陈宁江, 章德华. 基于特征投影与自适应增强的联邦蒸馏方法[J/OL]. 通信学报, 2026. DOI： 10.11959/j.issn.1000-436x.TXXB260116.

Chen Ningjiang, Zhang Dehua. A Federated Distillation Method Based on Feature Projection and Adaptive Enhancement[J/OL]. Journal on Communications, 2026. DOI： 10.11959/j.issn.1000-436x.TXXB260116.

摘要

为了解决现有联邦蒸馏方法难以处理异构模型间表征空间不一致与特征分布不均的问题，提出一种基于特征投影与自适应增强的联邦异构知识蒸馏框架，实现了跨异构客户端模型的知识高效融合。该框架在服务器端通过蒸馏方式整合客户端输出，在客户端设计轻量化多出口分支，将异构模型的中间特征投影到对齐的logits空间，以突破异构蒸馏中的特征对齐瓶颈。实验结果表明，该方法在标准数据集上取得良好效果，减少了通信轮次和数据传输量，同时增强了系统在异构环境下的鲁棒性，为联邦异构知识融合提供了一种有效的新方案。

Abstract

To address the problem that existing federated distillation methods struggle to handle inconsistent representation spaces and uneven feature distributions across heterogeneous models

a federated heterogeneous knowledge distillation framework based on feature projection and adaptive enhancement was proposed

achieving efficient knowledge fusion across heterogeneous client models. On the server side

client outputs were integrated via distillation; on the client side

lightweight multi-exit branches were employed to project intermediate features into an aligned logits space to overcome feature alignment bottlenecks. Promising performance was demonstrated on benchmark datasets

with reduced communication rounds and data transmission as well as enhanced system robustness

thus providing an effective new solution for heterogeneous model knowledge fusion in federated learning.

关键词

Keywords

references

Verbraeken J , Wolting M , Katzy J , et al . A survey on distributed machine learning [J ] . ACM Computing Surveys , 2020 , 53 ( 2 ): 1 - 33 .

Kairouz P , Mcmahan H B , Avent B , et al . Advances and open problems in federated learning [J ] . Foundations and Trends in Machine Learning , 2021 , 14 ( 1-2 ): 1 - 210 .

Mcmahan H B , Moore E , Ramage D , et al . Communication efficient learning of deep networks from decentralized data [C ] // Proc of the 20th Int Conf on Artificial Intelligence and Statistics . Fort Lauderdale , 2017 : 1273 - 1282 .

Jeong E , Oh S , Kim H , et al . Communication-efficient on device machine learning: Federated distillation and augmentation under non-IID private data [J/OL ] . arXiv preprint , 2018 : arXiv: 1811.11479 .

Li L , Gou J P , Yu B S , et al . Federated Distillation: A Survey [J/OL ] . arXiv preprint , 2024 : arXiv: 2404.08564 .

Ge H H , Pokhrel S R , Liu Z Y , et al . PFL-DKD: Modeling decoupled knowledge fusion with distillation for improving personalized federated learning [J ] . Computer Networks , 2024 , 254 : 110758 .

Liu W , Chen J Y , Wang B , et al . FedDM: A discrepancy-aware federated learning method based on multibranch feature fusion for non-IID data environments [J ] . IEEE Internet of Things Journal , 2025 , 12 ( 21 ): 45825 - 45835 .

Gou J P , Yu B , Maybank S J , et al . Knowledge distillation: a survey [J ] . International Journal of Computer Vision , 2021 , 129 ( 6 ): 1789 - 1819 .

Huang Y L , Hu K , Zhang Y , et al . Distilling knowledge from heterogeneous architectures for semantic segmentation [C ] // Proc of the AAAI Conference on Artificial Intelligence . Vancouver , 2024 , 38 : 3824 - 3832 .

Li Y C , Wang X Y , Xu W C , et al . Feature distillation is the better choice for model-heterogeneous federated learning [C ] // Proc of the 39th Advances in Neural Information Processing Systems . Vancouver , 2025 : 104726 - 104744 .

Lin T , Kong L J , Stich S U , et al . Ensemble distillation for robust model fusion in federated learning [C ] // Proc of NeurIPS 2020 . Vancouver , 2020 : 2351 - 2363 .

郑晶晶 , 赖金山 , 张凤荔 , 等 . FedASHP：高性能通信的自适应稀疏高效联邦学习方法 [J/OL ] . 小型微型计算机系统 , 1 - 9 [ 2026-04-15 ] . https://doi.org/10.20009/j.cnki.21-1106/TP.2025-0342 https://doi.org/10.20009/j.cnki.21-1106/TP.2025-0342 .

Zheng J J , Lai J S , Zhang F L , et al . FedASHP: An adaptive sparse efficient federated learning method for high-performance communication [J/OL ] . Journal of Chinese Computer Systems , 1 - 9 [ 2026-04-15 ] . https://doi.org/10.20009/j.cnki.21-1106/TP.2025-0342 https://doi.org/10.20009/j.cnki.21-1106/TP.2025-0342 .

刘佳瑜 , 王勇 , 杨静 , 等 . FedDW：基于一致性优化的权重蒸馏异构联邦学习方法 [J ] . 计算机研究与发展 , 2026 , 63 ( 4 ): 998 - 1009 .

Liu J Y , Wang Y , Yang J , et al . FedDW: Weight distillation heterogeneous federated learning method based on consensus optimization [J ] . Journal of Computer Research and Development , 2026 , 63 ( 4 ): 998 - 1009 .

王瑞锦 , 王金波 , 张凤荔 , 等 . 联邦原型学习的特征图中毒攻击和双重防御机制 [J ] . 软件学报 , 2025 , 36 ( 3 ): 1355 - 1374 .

Wang R J , Wang J B , Zhang F L , et al . Feature graph poisoning attack and double defense mechanism for federated prototype learning [J ] . Journal of Software , 2025 , 36 ( 3 ): 1355 - 1374 .

Siddika F , Hossen M A , Zhang W S , et al . Dual-Distilled Heterogeneous Federated Learning with Adaptive Margins for Trainable Global Prototypes [C ] // Proc of the 42nd Int Conf on Machine Learning . Vienna , 2025 .

Song W S , Yan M W , Li X Z , et al . Bidirectional decoupled distillation for heterogeneous federated learning [J ] . Entropy , 2024 , 26 ( 9 ): 762 .

Li K , Ding Y , Zhu Z Q , et al . Transforming gaps into gains: bridging model and data heterogeneity in federated learning via knowledge weak-aware zones [C ] // Proc of the 39th Advances in Neural Information Processing Systems . Vancouver , 2025 .

Yu Y , Kim N . Heterogeneous knowledge distillation using conceptual learning [J ] . IEEE Access , 2024 , 12 : 52803 - 52814 .

Li T , Sahu A K , Talwalkar A , et al . Federated learning: challenges, methods, and future directions [J ] . IEEE Signal Processing Magazine , 2020 , 37 ( 3 ): 50 - 60 .

Bottou L , Curtis F E , Nocedal J . Optimization methods for large-scale machine learning [J ] . SIAM Review , 2018 , 60 ( 2 ): 223 - 311 .

Hsu T M H , Qi H , Brown M . Measuring the effects of non-identical data distribution for federated visual classification [J/OL ] . arXiv preprint arXiv: 1909.06335 , 2019 .

Li T , Sahu A K , Zaheer M , et al . FedProx: Federated optimization in heterogeneous networks [C ] // Proc of MLSys 2020 . Austin , 2020 : 429 - 450 .

Li Q B , He B S , Song D . Model-contrastive federated learning [C ] // Proc of the IEEE/CVF Conf on Computer Vision and Pattern Recognition . Nashville , 2021 : 10713 - 10722 .

Wu C , Wu F , Lyu L , et al . Communication-efficient federated learning via knowledge distillation [J ] . Nature Communications , 2022 , 13 ( 1 ): 2032 .

Tan Y , Long G D , Liu L , et al . FedProto: Federated prototype learning across heterogeneous clients [C ] // Proc of AAAI 2022 . Vancouver , 2022 : 8042 - 8050 .

Chen H Y , Chao W L . On bridging generic and personalized federated learning for image classification [C ] // Proc of the 10th Int Conf on Learning Representations . Virtual , 2022 .

Yi L P , Yu H , Ren C , et al . Federated Model Heterogeneous Matryoshka Representation Learning [C ] // Proc of the 38th Advances in Neural Information Processing Systems . Vancouver , 2024 : 66431 - 66454 .

Zhang J Q , Liu Y , Hua Y , et al . FedTGP: Trainable Global Prototypes with Adaptive-Margin-Enhanced Contrastive Learning for Data and Model Heterogeneity in Federated Learning [C ] // Proc of the 38th AAAI Conference on Artificial Intelligence . AAAI Press , 2024 : 16768 - 16776 .

Du H Z , Xiang Y R , Cai Y W , et al . FedFree: Breaking Knowledge-sharing Barriers through Layer-wise Alignment in Heterogeneous Federated Learning [C ] // The 39th Annual Conference on Neural Information Processing Systems . Vancouver, Canada : Neural Information Processing Systems Foundation , 2025 .

Han P , Xiao H , Zheng S , et al . FedMKD: Hybrid Feature Guided Multilayer Fusion Knowledge Distillation in Heterogeneous Federated Learning [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2026 , 37 ( 3 ): 1463 - 1476 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

ISAC网络中基于特征信息的联邦学习动态压缩方法

DPBR-Adapt：具有层级自适应差分隐私的联邦学习防御方案

面向协作频谱感知的个性化差分隐私联邦学习方法

面向动态算力节点的联邦学习差分隐私重校准方法

HFLGuard：基于聚类和自编码器的异构联邦中毒防御方法