Node selection method in federated learning based on deep reinforcement learning

Wenchen HE; Shaoyong GUO; Xuesong QIU; Liandong CHEN; Suxiang ZHANG

doi:10.11959/j.issn.1000-436x.2021111

您当前的位置：

首页 >

文章列表页 >

Node selection method in federated learning based on deep reinforcement learning

Papers | 更新时间：2024-06-05

- Node selection method in federated learning based on deep reinforcement learning
- Journal on Communications Vol. 42, Issue 6, Pages: 62-71(2021)
- 作者机构：
  
  1. 北京邮电大学网络与交换技术国家重点实验室，北京 100876
  2. 国网河北信息通信分公司，河北石家庄 050011
  3. 国家电网有限公司信息通信分公司，北京 100761
- 作者简介：
- 基金信息：
  
  The National Natural Science Foundation of China(62071070);Ministry of Education Blockchain Core Project(2020KJ010802);The Key Research and Development Program of Hebei Province(20310103D)
- DOI：10.11959/j.issn.1000-436x.2021111
  CLC： TP911.1
- Online First：2021-06，
  
  Published：25 June 2021
- 稿件说明：
移动端阅览
Wenchen HE, Shaoyong GUO, Xuesong QIU, et al. Node selection method in federated learning based on deep reinforcement learning[J]. Journal on Communications, 2021, 42(6): 62-71.
DOI：

Wenchen HE, Shaoyong GUO, Xuesong QIU, et al. Node selection method in federated learning based on deep reinforcement learning[J]. Journal on Communications, 2021, 42(6): 62-71. DOI： 10.11959/j.issn.1000-436x.2021111.

摘要

为了应对设备差异化计算能力及非独立同分布数据对联邦学习性能的影响，高效地调度终端设备完成模型聚合，提出了一种基于深度强化学习的设备节点选择方法。该方法考虑异构节点的训练质量和效率，筛选恶意节点，在提升联邦学习模型准确率的同时，优化训练时延。首先，根据联邦学习中模型分布式训练的特点，构建基于深度强化学习的节点选择系统模型。其次，考虑设备训练时延、模型传输时延和准确率等因素，提出面向节点选择的准确率最优化问题模型。然后，将问题模型构建为马尔可夫决策过程，并设计基于分布式近端策略优化的节点选择算法，在每次训练迭代前选择合理的设备集合完成模型聚合。仿真实验表明，所提方法显著提高了联邦学习的准确率和训练速度，且具有良好的收敛性和稳健性。

Abstract

To cope with the impact of different device computing capabilities and non-independent uniformly distributed data on federated learning performance

and to efficiently schedule terminal devices to complete model aggregation

a method of node selection based on deep reinforcement learning was proposed.It considered training quality and efficiency of heterogeneous terminal devices

and filtrate malicious nodes to guarantee higher model accuracy and shorter training delay of federated learning.Firstly

according to characteristics of model distributed training in federated learning

a node selection system model based on deep reinforcement learning was constructed.Secondly

considering such factors as device training delay

model transmission delay and accuracy

an optimization model of accuracy for node selection was proposed.Finally

the problem model was constructed as a Markov decision process and a node selection algorithm based on distributed proximal strategy optimization was designed to obtain a reasonable set of devices before each training iteration to complete model aggregation.Simulation results demonstrate that the proposed method significantly improves the accuracy and training speed of federated learning

and its convergence and robustness are also well.

关键词

Keywords

references

ZHOU Z , CHEN X , LI E , et al . Edge intelligence:paving the last mile of artificial intelligence with edge computing [J ] . Proceedings of the IEEE , 2019 , 107 ( 8 ): 1738 - 1762 .

LU Y L , HUANG X H , DAI Y Y , et al . Federated learning for data privacy preservation in vehicular cyber-physical systems [J ] . IEEE Network , 2020 , 34 ( 3 ): 50 - 56 .

陈兵 , 成翔 , 张佳乐 , 等 . 联邦学习安全与隐私保护综述 [J ] . 南京航空航天大学学报 , 2020 , 52 ( 5 ): 675 - 684 .

CHEN B , CHENG X , ZHANG J L , et al . Survey of security and privacy in federated learning [J ] . Journal of Nanjing University of Aeronautics ＆ Astronautics , 2020 , 52 ( 5 ): 675 - 684 .

WANG H , KAPLAN Z , NIU D , et al . Optimizing federated learning on non-IID data with reinforcement learning [C ] // IEEE INFOCOM 2020 - IEEE Conference on Computer Communications . Piscataway:IEEE Press , 2020 : 1698 - 1707 .

ABDULRAHMAN S , TOUT H ， OULD-SLIMANE H , et al . A survey on federated learning:the journey from centralized to distributed on-site learning and beyond [J ] . IEEE Internet of Things Journal , 2021 , 8 ( 7 ): 5476 - 5497 .

SHI W Q , ZHOU S , NIU Z S . Device scheduling with fast convergence for wireless federated learning [C ] // 2020 IEEE International Conference on Communications . Piscataway:IEEE Press , 2020 : 1 - 6 .

REN J K , HE Y H , WEN D Z , et al . Scheduling for cellular federated edge learning with importance and channel awareness [J ] . IEEE Transactions on Wireless Communications , 2020 , 19 ( 11 ): 7690 - 7703 .

CHEN M Z , POOR H V , SAAD W , et al . Convergence time minimization of federated learning over wireless networks [C ] // 2020 IEEE International Conference on Communications . Piscataway:IEEE Press , 2020 : 1 - 6 .

WU W T , HE L G , LIN W W , et al . Accelerating federated learning over reliability-agnostic clients in mobile edge computing systems [J ] . IEEE Transactions on Parallel and Distributed Systems , 2021 , 32 ( 7 ): 1539 - 1551 .

KANG J W , XIONG Z H , NIYATO D , et al . Incentive mechanism for reliable federated learning:a joint optimization approach to combining reputation and contract theory [J ] . IEEE Internet of Things Journal , 2019 , 6 ( 6 ): 10700 - 10714 .

LU Y L , HUANG X H , ZHANG K , et al . Blockchain empowered asynchronous federated learning for secure data sharing in Internet of vehicles [J ] . IEEE Transactions on Vehicular Technology , 2020 , 69 ( 4 ): 4298 - 4311 .

YOSHIDA N , NISHIO T , MORIKURA M , et al . Hybrid-FL for wireless networks:cooperative learning mechanism using non-IID data [C ] // 2020 IEEE International Conference on Communications . Piscataway:IEEE Press , 2020 : 1 - 7 .

YANG Z H , CHEN M Z , SAAD W , et al . Energy efficient federated learning over wireless communication networks [J ] . IEEE Transactions on Wireless Communications , 2021 , 20 ( 3 ): 1935 - 1949 .

ZENG T C , SEMIARI O , MOZAFFARI M , et al . Federated learning in the sky:joint power allocation and scheduling with UAV swarms [C ] // 2020 IEEE International Conference on Communications . Piscataway:IEEE Press , 2020 : 1 - 6 .

TRAN H V , KADDOUM G , ELGALA H , et al . Lightwave power transfer for federated learning-based wireless networks [J ] . IEEE Communications Letters , 2020 , 24 ( 7 ): 1472 - 1476 .

LUO S Q , CHEN X , WU Q , et al . HFEL:joint edge association and resource allocation for cost-efficient hierarchical federated edge learning [J ] . IEEE Transactions on Wireless Communications , 2020 , 19 ( 10 ): 6535 - 6548 .

孟洛明 , 孙康 , 韦磊 , 等 . 一种面向电力无线专网的虚拟资源优化分配机制 [J ] . 电子与信息学报 , 2017 , 39 ( 7 ): 1711 - 1718 .

MENG L M , SUN K , WEI L , et al . Optimal resource allocation mechanism for electric power wireless virtual networks [J ] . Journal of Electronics ＆ Information Technology , 2017 , 39 ( 7 ): 1711 - 1718 .

李枝灵 , 刘柱 , 郭少勇 , 等 . 基于免疫算法的电力线通信网接入点规划方法 [J ] . 北京邮电大学学报 , 2016 , 39 ( S1 ): 104 - 108 .

LI Z L , LIU Z , GUO S Y , et al . Access points location planning based on immune algorithm for power line communication network [J ] . Journal of Beijing University of Posts and Telecommunications , 2016 , 39 ( S1 ): 104 - 108 .

赵海涛 , 张唐伟 , 陈跃 , 等 . 基于DQN的车载边缘网络任务分发卸载算法 [J ] . 通信学报 , 2020 , 41 ( 10 ): 172 - 178 .

ZHAO H T , ZHANG T W , CHEN Y , et al . Task distribution offloading algorithm of vehicle edge network based on DQN [J ] . Journal on Communications , 2020 , 41 ( 10 ): 172 - 178 .

喻鹏 , 张俊也 , 李文璟 , 等 . 移动边缘网络中基于双深度 Q 学习的高能效资源分配方法 [J ] . 通信学报 , 2020 , 41 ( 12 ): 148 - 161 .

YU P , ZHANG J Y , LI W J , et al . Energy-efficient resource allocation method in mobile edge network based on double deep Q-learning [J ] . Journal on Communications , 2020 , 41 ( 12 ): 148 - 161 .

PAN S L , ZHANG Z Y , ZHANG Z W , et al . Dependency-aware computation offloading in mobile edge computing:a reinforcement learning approach [J ] . IEEE Access , 2019 , 7 : 134742 - 134753 .

YANG Z Y , MERRICK K , JIN L W , et al . Hierarchical deep reinforcement learning for continuous action control [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2018 , 29 ( 11 ): 5174 - 5184 .

LIANG X Y , DU X S , WANG G L , et al . A deep reinforcement learning network for traffic light cycle control [J ] . IEEE Transactions on Vehicular Technology , 2019 , 68 ( 2 ): 1243 - 1253 .

LIU C F , BENNIS M , DEBBAH M , et al . Dynamic task offloading and resource allocation for ultra-reliable low-latency edge computing [J ] . IEEE Transactions on Communications , 2019 , 67 ( 6 ): 4132 - 4150 .

SCHULMAN J , LEVINE S , MORITZ P , et al . Trust region policy optimization [J ] . arXiv Preprint,arXiv:1502.05477 , 2015 .

Views

1624

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Communication-efficient federated learning method via redundant data elimination

Overview on intelligent wireless communication technology

Dynamic compression method for federated learning based on feature information in ISAC networks

DPBR-Adapt: a hierarchically adaptive differential privacy defence scheme for federated learning

Personalized differential privacy federated learning method for collaborative spectrum sensing

Related Author

Kaiju LI

Qiang XU

Hao WANG

Yingchang LIANG

Junjie TAN

Dusit Niyato

Deng Bingguang

Peng Jiayin

Related Institution

School of Computer Science and Technology, Chongqing University of Posts and Telecommunications

College of Computer Science, Chongqing University

Department of Electrical Engineering, City University of Hong Kong

Key Laboratory of Tourism Multisource Data Perception and Decision, Ministry of Culture and Tourism, Chongqing University of Posts and Telecommunications

National Key Laboratory of Science and Technology on Communications,University of Electronic Science and Technology of China

AI问答

⁰