基于DRL的联邦学习节点选择方法

贺文晨; 郭少勇; 邱雪松; 陈连栋; 张素香

doi:10.11959/j.issn.1000-436x.2021111

您当前的位置：

首页 >

文章列表页 >

基于DRL的联邦学习节点选择方法

学术论文 | 更新时间：2024-06-05

- 基于DRL的联邦学习节点选择方法
- Node selection method in federated learning based on deep reinforcement learning
- 通信学报 2021年42卷第6期页码：62-71
- 作者机构：
  
  1. 北京邮电大学网络与交换技术国家重点实验室，北京 100876
  2. 国网河北信息通信分公司，河北石家庄 050011
  3. 国家电网有限公司信息通信分公司，北京 100761
- 作者简介：
  
  [ "贺文晨（1993− ），男，山东济南人，北京邮电大学博士生，主要研究方向为边缘智能、任务部署和资源分配" ]
  [ "郭少勇（1985− ），男，河北邢台人，博士，北京邮电大学副教授，主要研究方向为物联网与区块链" ]
  [ "邱雪松（1973− ），男，江西上饶人，博士，北京邮电大学教授、博士生导师，主要研究方向为网络与业务管理、物联网与区块链" ]
  [ "陈连栋（1987− ），男，山东莱州人，国网河北信息通信分公司副高级工程师，主要研究方向为网络信息安全和数据加密" ]
  [ "张素香（1973− ），女，河北衡水人，博士，国家电网有限公司信息通信分公司高级工程师，主要研究方向为电力系统通信、物联网和人工智能等" ]
- 基金信息：
  
  国家自然科学基金资助项目(62071070);教育部区场链核心计划基金资助项目(2020KJ010802);河北省重点研发计划基金资助项目(20310103D)
- DOI：10.11959/j.issn.1000-436x.2021111
  中图分类号： TP911.1
- 网络首发：2021-06，
  
  纸质出版：2021-06-25
- 稿件说明：
移动端阅览
贺文晨, 郭少勇, 邱雪松, 等. 基于DRL的联邦学习节点选择方法[J]. 通信学报, 2021,42(6):62-71.

Wenchen HE, Shaoyong GUO, Xuesong QIU, et al. Node selection method in federated learning based on deep reinforcement learning[J]. Journal on Communications, 2021, 42(6): 62-71.
贺文晨, 郭少勇, 邱雪松, 等. 基于DRL的联邦学习节点选择方法[J]. 通信学报, 2021,42(6):62-71. DOI： 10.11959/j.issn.1000-436x.2021111.

Wenchen HE, Shaoyong GUO, Xuesong QIU, et al. Node selection method in federated learning based on deep reinforcement learning[J]. Journal on Communications, 2021, 42(6): 62-71. DOI： 10.11959/j.issn.1000-436x.2021111.

摘要

为了应对设备差异化计算能力及非独立同分布数据对联邦学习性能的影响，高效地调度终端设备完成模型聚合，提出了一种基于深度强化学习的设备节点选择方法。该方法考虑异构节点的训练质量和效率，筛选恶意节点，在提升联邦学习模型准确率的同时，优化训练时延。首先，根据联邦学习中模型分布式训练的特点，构建基于深度强化学习的节点选择系统模型。其次，考虑设备训练时延、模型传输时延和准确率等因素，提出面向节点选择的准确率最优化问题模型。然后，将问题模型构建为马尔可夫决策过程，并设计基于分布式近端策略优化的节点选择算法，在每次训练迭代前选择合理的设备集合完成模型聚合。仿真实验表明，所提方法显著提高了联邦学习的准确率和训练速度，且具有良好的收敛性和稳健性。

Abstract

To cope with the impact of different device computing capabilities and non-independent uniformly distributed data on federated learning performance

and to efficiently schedule terminal devices to complete model aggregation

a method of node selection based on deep reinforcement learning was proposed.It considered training quality and efficiency of heterogeneous terminal devices

and filtrate malicious nodes to guarantee higher model accuracy and shorter training delay of federated learning.Firstly

according to characteristics of model distributed training in federated learning

a node selection system model based on deep reinforcement learning was constructed.Secondly

considering such factors as device training delay

model transmission delay and accuracy

an optimization model of accuracy for node selection was proposed.Finally

the problem model was constructed as a Markov decision process and a node selection algorithm based on distributed proximal strategy optimization was designed to obtain a reasonable set of devices before each training iteration to complete model aggregation.Simulation results demonstrate that the proposed method significantly improves the accuracy and training speed of federated learning

and its convergence and robustness are also well.

关键词

Keywords

references

ZHOU Z , CHEN X , LI E , et al . Edge intelligence:paving the last mile of artificial intelligence with edge computing [J ] . Proceedings of the IEEE , 2019 , 107 ( 8 ): 1738 - 1762 .

LU Y L , HUANG X H , DAI Y Y , et al . Federated learning for data privacy preservation in vehicular cyber-physical systems [J ] . IEEE Network , 2020 , 34 ( 3 ): 50 - 56 .

陈兵 , 成翔 , 张佳乐 , 等 . 联邦学习安全与隐私保护综述 [J ] . 南京航空航天大学学报 , 2020 , 52 ( 5 ): 675 - 684 .

CHEN B , CHENG X , ZHANG J L , et al . Survey of security and privacy in federated learning [J ] . Journal of Nanjing University of Aeronautics ＆ Astronautics , 2020 , 52 ( 5 ): 675 - 684 .

WANG H , KAPLAN Z , NIU D , et al . Optimizing federated learning on non-IID data with reinforcement learning [C ] // IEEE INFOCOM 2020 - IEEE Conference on Computer Communications . Piscataway:IEEE Press , 2020 : 1698 - 1707 .

ABDULRAHMAN S , TOUT H ， OULD-SLIMANE H , et al . A survey on federated learning:the journey from centralized to distributed on-site learning and beyond [J ] . IEEE Internet of Things Journal , 2021 , 8 ( 7 ): 5476 - 5497 .

SHI W Q , ZHOU S , NIU Z S . Device scheduling with fast convergence for wireless federated learning [C ] // 2020 IEEE International Conference on Communications . Piscataway:IEEE Press , 2020 : 1 - 6 .

REN J K , HE Y H , WEN D Z , et al . Scheduling for cellular federated edge learning with importance and channel awareness [J ] . IEEE Transactions on Wireless Communications , 2020 , 19 ( 11 ): 7690 - 7703 .

CHEN M Z , POOR H V , SAAD W , et al . Convergence time minimization of federated learning over wireless networks [C ] // 2020 IEEE International Conference on Communications . Piscataway:IEEE Press , 2020 : 1 - 6 .

WU W T , HE L G , LIN W W , et al . Accelerating federated learning over reliability-agnostic clients in mobile edge computing systems [J ] . IEEE Transactions on Parallel and Distributed Systems , 2021 , 32 ( 7 ): 1539 - 1551 .

KANG J W , XIONG Z H , NIYATO D , et al . Incentive mechanism for reliable federated learning:a joint optimization approach to combining reputation and contract theory [J ] . IEEE Internet of Things Journal , 2019 , 6 ( 6 ): 10700 - 10714 .

LU Y L , HUANG X H , ZHANG K , et al . Blockchain empowered asynchronous federated learning for secure data sharing in Internet of vehicles [J ] . IEEE Transactions on Vehicular Technology , 2020 , 69 ( 4 ): 4298 - 4311 .

YOSHIDA N , NISHIO T , MORIKURA M , et al . Hybrid-FL for wireless networks:cooperative learning mechanism using non-IID data [C ] // 2020 IEEE International Conference on Communications . Piscataway:IEEE Press , 2020 : 1 - 7 .

YANG Z H , CHEN M Z , SAAD W , et al . Energy efficient federated learning over wireless communication networks [J ] . IEEE Transactions on Wireless Communications , 2021 , 20 ( 3 ): 1935 - 1949 .

ZENG T C , SEMIARI O , MOZAFFARI M , et al . Federated learning in the sky:joint power allocation and scheduling with UAV swarms [C ] // 2020 IEEE International Conference on Communications . Piscataway:IEEE Press , 2020 : 1 - 6 .

TRAN H V , KADDOUM G , ELGALA H , et al . Lightwave power transfer for federated learning-based wireless networks [J ] . IEEE Communications Letters , 2020 , 24 ( 7 ): 1472 - 1476 .

LUO S Q , CHEN X , WU Q , et al . HFEL:joint edge association and resource allocation for cost-efficient hierarchical federated edge learning [J ] . IEEE Transactions on Wireless Communications , 2020 , 19 ( 10 ): 6535 - 6548 .

孟洛明 , 孙康 , 韦磊 , 等 . 一种面向电力无线专网的虚拟资源优化分配机制 [J ] . 电子与信息学报 , 2017 , 39 ( 7 ): 1711 - 1718 .

MENG L M , SUN K , WEI L , et al . Optimal resource allocation mechanism for electric power wireless virtual networks [J ] . Journal of Electronics ＆ Information Technology , 2017 , 39 ( 7 ): 1711 - 1718 .

李枝灵 , 刘柱 , 郭少勇 , 等 . 基于免疫算法的电力线通信网接入点规划方法 [J ] . 北京邮电大学学报 , 2016 , 39 ( S1 ): 104 - 108 .

LI Z L , LIU Z , GUO S Y , et al . Access points location planning based on immune algorithm for power line communication network [J ] . Journal of Beijing University of Posts and Telecommunications , 2016 , 39 ( S1 ): 104 - 108 .

赵海涛 , 张唐伟 , 陈跃 , 等 . 基于DQN的车载边缘网络任务分发卸载算法 [J ] . 通信学报 , 2020 , 41 ( 10 ): 172 - 178 .

ZHAO H T , ZHANG T W , CHEN Y , et al . Task distribution offloading algorithm of vehicle edge network based on DQN [J ] . Journal on Communications , 2020 , 41 ( 10 ): 172 - 178 .

喻鹏 , 张俊也 , 李文璟 , 等 . 移动边缘网络中基于双深度 Q 学习的高能效资源分配方法 [J ] . 通信学报 , 2020 , 41 ( 12 ): 148 - 161 .

YU P , ZHANG J Y , LI W J , et al . Energy-efficient resource allocation method in mobile edge network based on double deep Q-learning [J ] . Journal on Communications , 2020 , 41 ( 12 ): 148 - 161 .

PAN S L , ZHANG Z Y , ZHANG Z W , et al . Dependency-aware computation offloading in mobile edge computing:a reinforcement learning approach [J ] . IEEE Access , 2019 , 7 : 134742 - 134753 .

YANG Z Y , MERRICK K , JIN L W , et al . Hierarchical deep reinforcement learning for continuous action control [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2018 , 29 ( 11 ): 5174 - 5184 .

LIANG X Y , DU X S , WANG G L , et al . A deep reinforcement learning network for traffic light cycle control [J ] . IEEE Transactions on Vehicular Technology , 2019 , 68 ( 2 ): 1243 - 1253 .

LIU C F , BENNIS M , DEBBAH M , et al . Dynamic task offloading and resource allocation for ultra-reliable low-latency edge computing [J ] . IEEE Transactions on Communications , 2019 , 67 ( 6 ): 4132 - 4150 .

SCHULMAN J , LEVINE S , MORITZ P , et al . Trust region policy optimization [J ] . arXiv Preprint,arXiv:1502.05477 , 2015 .

浏览量

1598

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

冗余数据去除的联邦学习高效通信方法

智能无线通信技术研究概况

面向动态算力节点的联邦学习差分隐私重校准方法

HFLGuard：基于聚类和自编码器的异构联邦中毒防御方法

基于DDPG强化学习的多集群算力资源调度算法