MapReduce框架下支持差分隐私保护的随机梯度下降算法
Stochastic gradient descent algorithm preserving differential privacy in MapReduce framework
- 通信学报 2018年39卷第1期 页码：70-77
- 作者机构：
海军工程大学信息安全系，湖北 武汉 430033
- 作者简介：
[ "俞艺涵（1992-），男，浙江金华人，海军工程大学博士生，主要研究方向为信息系统安全、隐私保护等。" ]
[ "付钰（1982-），女，湖北武汉人，博士，海军工程大学副教授、硕士生导师，主要研究方向为信息安全风险评估等。" ]
[ "吴晓平（1961-），男，山西新绛人，博士，海军工程大学教授、博士生导师，主要研究方向为信息安全、密码学等。" ]
- 基金信息：国家自然科学基金青年基金资助项目(61100042);国家社科基金资助项目(15GJ003-201)
DOI：10.11959/j.issn.1000-436x.2018013
中图分类号： TP301-
网络首发：2018-01，
纸质出版：2018-01-25
- 稿件说明：
移动端阅览
俞艺涵, 付钰, 吴晓平. MapReduce框架下支持差分隐私保护的随机梯度下降算法[J]. 通信学报, 2018,39(1):70-77.
Yihan YU, Yu FU, Xiaoping WU. Stochastic gradient descent algorithm preserving differential privacy in MapReduce framework[J]. Journal on Communications, 2018, 39(1): 70-77.
俞艺涵, 付钰, 吴晓平. MapReduce框架下支持差分隐私保护的随机梯度下降算法[J]. 通信学报, 2018,39(1):70-77. DOI： 10.11959/j.issn.1000-436x.2018013.
Yihan YU, Yu FU, Xiaoping WU. Stochastic gradient descent algorithm preserving differential privacy in MapReduce framework[J]. Journal on Communications, 2018, 39(1): 70-77. DOI： 10.11959/j.issn.1000-436x.2018013.
摘要
针对现有分布式计算环境下随机梯度下降算法存在效率性与私密性矛盾的问题，提出一种 MapReduce框架下满足差分隐私的随机梯度下降算法。该算法基于MapReduce框架，将数据随机分配到各个Map节点并启动Map分任务独立并行执行随机梯度下降算法；启动Reduce分任务合并满足更新要求的分目标更新模型，并加入拉普拉斯随机噪声实现差分隐私保护。根据差分隐私保护原理，证明了算法满足ε-差分隐私保护要求。实验表明该算法具有明显的效率优势并有较好的数据可用性。
Abstract
Aiming at the contradiction between the efficiency and privacy of stochastic gradient descent algorithm in distributed computing environment
a stochastic gradient descent algorithm preserving differential privacy based on MapReduce was proposed.Based on the computing framework of MapReduce
the data were allocated randomly to each Map node and the Map tasks were started independently to execute the stochastic gradient descent algorithm.The Reduce tasks were appointed to update the model when the sub-target update models were meeting the update requirements
and to add Laplace random noise to achieve differential privacy protection.Based on the combinatorial features of differential privacy
the results of the algorithm is proved to be able to fulfill ε-differentially private.The experimental results show that the algorithm has obvious efficiency advantage and good data availability.
关键词
Keywords
references
WU F , LI F G , KUMAR A , et al . Bolt-on differential privacy for scalable stochastic gradient descent-based analytics [C ] // The 2017 ACM International Conference on Management of Data . 2017 : 1307 - 1322 .
ABADI M , CHU A , GOODFELLOW I , et al . Deep learning with differential privacy [C ] // The 2016 ACM SIGSAC Conference on Computer and Communications Security . 2016 : 308 - 318 .
ZHAO P , ZHANG T . Stochastic optimization with importance sampling [J ] . Eprint Arxiv , 2015 : 1 - 9 .
SCHMIDT M , ROUX N L , BACH F . Erratum to:minimizing finite sums with the stochastic average gradient [J ] . Mathematical Programming , 2016 , 162 ( 5 ):1.
MU Y , LIU W , LIU X , et al . Stochastic gradient made stable:a manifold propagation approach for large-scale optimization [J ] . IEEE Transactions on Knowledge ＆ Data Engineering , 2015 , 29 ( 2 ): 458 - 471 .
ZINKEVICH M , WEIMER M , SMOLA A J , et al . Parallelized stochastic gradient descent [C ] // The Conference on Neural Information Processing Systems . 2011 : 2595 - 2603 .
陈振宏 , 兰艳艳 , 郭嘉丰 , 等 . 基于差异合并的分布式随机梯度下降算法 [J ] . 计算机学报 , 2015 , 38 ( 10 ): 2054 - 2063 .
CHEN Z H , LAN Y Y , GUO J F , et al . Distributed stochastic gradient descent with discriminative aggregating [J ] . Chinese Journal of Computers , 2015 , 38 ( 10 ): 2054 - 2063 .
ZHAO H , CANNY J F . Communication-efficient distributed stochastic gradient descent with butterfly mixing [D ] . Berkeley,USA:University of California , 2012 .
SONG S , CHAUDHURI K , SARWATE A D . Stochastic gradient descent with differentially private updates [C ] // Global conference on Signal and Information Processing (GlobalSIP) . 2013 : 245 - 248 .
BASSILY R , THAKURTA A . Private empirical risk minimization:Efficient algorithms and tight error bounds [C ] // 2014 IEEE 55th Annual Symposium on Foundations of Computer Science (FOCS) . 2014 : 464 - 473 .
DWORK C , MCSHERRY F , NISSIM K . Calibrating noise to sensitivity in private data analysis [J ] . The VLDB Endowment , 2006 , 7 ( 8 ): 637 - 648 .
DWORK C , ROTH A . The Algorithmic foundations of differential privacy [M ] . Now Publishers Inc , 2014 .
CHAUDHURI K , MONTELEONI C , SARWATE A D . Differentially private empirical risk minimization [J ] . Journal of Machine Learning Research , 2009 , 12 ( 2 ): 1069 - 1109 .
何贤芒 , 王晓阳 , 陈华辉 , 等 . 差分隐私保护参数ε的选取研究 [J ] . 通信学报 , 2015 , 36 ( 12 ): 124 - 130 .
HE X M , WANG X Y , CHEN H H , et al . Study on choosing the parameter ε in differential privacy [J ] . Journal on Communications , 2015 , 36 ( 12 ): 124 - 130 .
MCSHERRY F D . Privacy integrated queries:an extensible platform for privacy-preserving data analysis [J ] . Communication of the ACM , 2010 , 53 ( 9 ): 89 - 97 .
0
浏览量
1759
下载量
1
CSCD
- 网络PDF下载
- XML下载
- Meta-XML下载
- BibTeX下载
- Endnote下载
- RefMan 下载
- NoteExpress下载
- NoteFirst下载
关联资源
相关文章
相关作者
相关机构
AI问答- 技术支持由北京北大方正电子有限公司提供 京ICP备09064830号-19
京公网安备11010802024621 - 本系统建议在Chrome、 IE9+ 以上版本浏览器阅读本站内容，360浏览器请切换至极速模式
- Cookies帮助我们提供服务并提供个性化体验。使用本网站，即表示您同意我们使用Cookies
- 总访问量：582505 今日访问量：321