Stochastic gradient descent algorithm preserving differential privacy in MapReduce framework

Yihan YU; Yu FU; Xiaoping WU

doi:10.11959/j.issn.1000-436x.2018013

您当前的位置：

首页 >

文章列表页 >

Stochastic gradient descent algorithm preserving differential privacy in MapReduce framework

Papers | 更新时间：2024-06-05

- Stochastic gradient descent algorithm preserving differential privacy in MapReduce framework
- Journal on Communications Vol. 39, Issue 1, Pages: 70-77(2018)
- 作者机构：
  
  海军工程大学信息安全系，湖北武汉 430033
- 作者简介：
- 基金信息：
  
  The National Natural Science Foundation of China (Project for Youth)(61100042);The National Social Science Foundation of China(15GJ003-201)
- DOI：10.11959/j.issn.1000-436x.2018013
  CLC： TP301
- Online First：2018-01，
  
  Published：25 January 2018
- 稿件说明：
移动端阅览
Yihan YU, Yu FU, Xiaoping WU. Stochastic gradient descent algorithm preserving differential privacy in MapReduce framework[J]. Journal on Communications, 2018, 39(1): 70-77.
DOI：

Yihan YU, Yu FU, Xiaoping WU. Stochastic gradient descent algorithm preserving differential privacy in MapReduce framework[J]. Journal on Communications, 2018, 39(1): 70-77. DOI： 10.11959/j.issn.1000-436x.2018013.

摘要

针对现有分布式计算环境下随机梯度下降算法存在效率性与私密性矛盾的问题，提出一种 MapReduce框架下满足差分隐私的随机梯度下降算法。该算法基于MapReduce框架，将数据随机分配到各个Map节点并启动Map分任务独立并行执行随机梯度下降算法；启动Reduce分任务合并满足更新要求的分目标更新模型，并加入拉普拉斯随机噪声实现差分隐私保护。根据差分隐私保护原理，证明了算法满足ε-差分隐私保护要求。实验表明该算法具有明显的效率优势并有较好的数据可用性。

Abstract

Aiming at the contradiction between the efficiency and privacy of stochastic gradient descent algorithm in distributed computing environment

a stochastic gradient descent algorithm preserving differential privacy based on MapReduce was proposed.Based on the computing framework of MapReduce

the data were allocated randomly to each Map node and the Map tasks were started independently to execute the stochastic gradient descent algorithm.The Reduce tasks were appointed to update the model when the sub-target update models were meeting the update requirements

and to add Laplace random noise to achieve differential privacy protection.Based on the combinatorial features of differential privacy

the results of the algorithm is proved to be able to fulfill ε-differentially private.The experimental results show that the algorithm has obvious efficiency advantage and good data availability.

关键词

Keywords

references

WU F , LI F G , KUMAR A , et al . Bolt-on differential privacy for scalable stochastic gradient descent-based analytics [C ] // The 2017 ACM International Conference on Management of Data . 2017 : 1307 - 1322 .

ABADI M , CHU A , GOODFELLOW I , et al . Deep learning with differential privacy [C ] // The 2016 ACM SIGSAC Conference on Computer and Communications Security . 2016 : 308 - 318 .

ZHAO P , ZHANG T . Stochastic optimization with importance sampling [J ] . Eprint Arxiv , 2015 : 1 - 9 .

SCHMIDT M , ROUX N L , BACH F . Erratum to:minimizing finite sums with the stochastic average gradient [J ] . Mathematical Programming , 2016 , 162 ( 5 ):1.

MU Y , LIU W , LIU X , et al . Stochastic gradient made stable:a manifold propagation approach for large-scale optimization [J ] . IEEE Transactions on Knowledge ＆ Data Engineering , 2015 , 29 ( 2 ): 458 - 471 .

ZINKEVICH M , WEIMER M , SMOLA A J , et al . Parallelized stochastic gradient descent [C ] // The Conference on Neural Information Processing Systems . 2011 : 2595 - 2603 .

陈振宏 , 兰艳艳 , 郭嘉丰 , 等 . 基于差异合并的分布式随机梯度下降算法 [J ] . 计算机学报 , 2015 , 38 ( 10 ): 2054 - 2063 .

CHEN Z H , LAN Y Y , GUO J F , et al . Distributed stochastic gradient descent with discriminative aggregating [J ] . Chinese Journal of Computers , 2015 , 38 ( 10 ): 2054 - 2063 .

ZHAO H , CANNY J F . Communication-efficient distributed stochastic gradient descent with butterfly mixing [D ] . Berkeley,USA:University of California , 2012 .

SONG S , CHAUDHURI K , SARWATE A D . Stochastic gradient descent with differentially private updates [C ] // Global conference on Signal and Information Processing (GlobalSIP) . 2013 : 245 - 248 .

BASSILY R , THAKURTA A . Private empirical risk minimization:Efficient algorithms and tight error bounds [C ] // 2014 IEEE 55th Annual Symposium on Foundations of Computer Science (FOCS) . 2014 : 464 - 473 .

DWORK C , MCSHERRY F , NISSIM K . Calibrating noise to sensitivity in private data analysis [J ] . The VLDB Endowment , 2006 , 7 ( 8 ): 637 - 648 .

DWORK C , ROTH A . The Algorithmic foundations of differential privacy [M ] . Now Publishers Inc , 2014 .

CHAUDHURI K , MONTELEONI C , SARWATE A D . Differentially private empirical risk minimization [J ] . Journal of Machine Learning Research , 2009 , 12 ( 2 ): 1069 - 1109 .

何贤芒 , 王晓阳 , 陈华辉 , 等 . 差分隐私保护参数ε的选取研究 [J ] . 通信学报 , 2015 , 36 ( 12 ): 124 - 130 .

HE X M , WANG X Y , CHEN H H , et al . Study on choosing the parameter ε in differential privacy [J ] . Journal on Communications , 2015 , 36 ( 12 ): 124 - 130 .

MCSHERRY F D . Privacy integrated queries:an extensible platform for privacy-preserving data analysis [J ] . Communication of the ACM , 2010 , 53 ( 9 ): 89 - 97 .

Views

1816

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

k-means clustering method preserving differential privacy in MapReduce framework

Fine-grained recognition method for encrypted FTP commands based on temporal dual-mode feature fusion

Survey on key technologies for native-intelligence-driven spectrum management towards 6G

Enhanced membership inference attack method leveraging data characteristics

Survey of IP geolocation: theory, methods, and innovative applications

Related Author

Hong-cheng LI

Xiao-ping WU

Yan CHEN

Fu Chunhui

Yang Zhi

Guo Yuanbo

Li Yongfei

Jin Shuyuan

Related Institution

No.61062 Troops of PLA

School of Computer Science and Engineering, Sun Yat-sen University

School of Cryptography, Information Engineering University

School of Cyberspace Security, Hainan University

Hebei Key Laboratory of Space-Air-Ground Intelligent Communication, University of Science and Technology Beijing

AI问答

⁰