MapReduce框架下支持差分隐私保护的随机梯度下降算法

俞艺涵; 付钰; 吴晓平

doi:10.11959/j.issn.1000-436x.2018013

您当前的位置：

首页 >

文章列表页 >

MapReduce框架下支持差分隐私保护的随机梯度下降算法

学术论文 | 更新时间：2024-06-05

- MapReduce框架下支持差分隐私保护的随机梯度下降算法
- Stochastic gradient descent algorithm preserving differential privacy in MapReduce framework
- 通信学报 2018年39卷第1期页码：70-77
- 作者机构：
  
  海军工程大学信息安全系，湖北武汉 430033
- 作者简介：
  
  [ "俞艺涵（1992-），男，浙江金华人，海军工程大学博士生，主要研究方向为信息系统安全、隐私保护等。" ]
  [ "付钰（1982-），女，湖北武汉人，博士，海军工程大学副教授、硕士生导师，主要研究方向为信息安全风险评估等。" ]
  [ "吴晓平（1961-），男，山西新绛人，博士，海军工程大学教授、博士生导师，主要研究方向为信息安全、密码学等。" ]
- 基金信息：
  
  国家自然科学基金青年基金资助项目(61100042);国家社科基金资助项目(15GJ003-201)
- DOI：10.11959/j.issn.1000-436x.2018013
  中图分类号： TP301
- 网络出版日期：2018-01，
  
  纸质出版日期：2018-01-25
- 稿件说明：
移动端阅览
俞艺涵, 付钰, 吴晓平. MapReduce框架下支持差分隐私保护的随机梯度下降算法[J]. 通信学报, 2018,39(1):70-77.

Yihan YU, Yu FU, Xiaoping WU. Stochastic gradient descent algorithm preserving differential privacy in MapReduce framework[J]. Journal on communications, 2018, 39(1): 70-77.
俞艺涵, 付钰, 吴晓平. MapReduce框架下支持差分隐私保护的随机梯度下降算法[J]. 通信学报, 2018,39(1):70-77. DOI： 10.11959/j.issn.1000-436x.2018013.

Yihan YU, Yu FU, Xiaoping WU. Stochastic gradient descent algorithm preserving differential privacy in MapReduce framework[J]. Journal on communications, 2018, 39(1): 70-77. DOI： 10.11959/j.issn.1000-436x.2018013.

摘要

针对现有分布式计算环境下随机梯度下降算法存在效率性与私密性矛盾的问题，提出一种 MapReduce框架下满足差分隐私的随机梯度下降算法。该算法基于MapReduce框架，将数据随机分配到各个Map节点并启动Map分任务独立并行执行随机梯度下降算法；启动Reduce分任务合并满足更新要求的分目标更新模型，并加入拉普拉斯随机噪声实现差分隐私保护。根据差分隐私保护原理，证明了算法满足ε-差分隐私保护要求。实验表明该算法具有明显的效率优势并有较好的数据可用性。

Abstract

Aiming at the contradiction between the efficiency and privacy of stochastic gradient descent algorithm in distributed computing environment

a stochastic gradient descent algorithm preserving differential privacy based on MapReduce was proposed.Based on the computing framework of MapReduce

the data were allocated randomly to each Map node and the Map tasks were started independently to execute the stochastic gradient descent algorithm.The Reduce tasks were appointed to update the model when the sub-target update models were meeting the update requirements

and to add Laplace random noise to achieve differential privacy protection.Based on the combinatorial features of differential privacy

the results of the algorithm is proved to be able to fulfill ε-differentially private.The experimental results show that the algorithm has obvious efficiency advantage and good data availability.

关键词

Keywords

references

WU F , LI F G , KUMAR A , et al . Bolt-on differential privacy for scalable stochastic gradient descent-based analytics [C ] // The 2017 ACM International Conference on Management of Data . 2017 : 1307 - 1322 .

ABADI M , CHU A , GOODFELLOW I , et al . Deep learning with differential privacy [C ] // The 2016 ACM SIGSAC Conference on Computer and Communications Security . 2016 : 308 - 318 .

ZHAO P , ZHANG T . Stochastic optimization with importance sampling [J ] . Eprint Arxiv , 2015 : 1 - 9 .

SCHMIDT M , ROUX N L , BACH F . Erratum to:minimizing finite sums with the stochastic average gradient [J ] . Mathematical Programming , 2016 , 162 ( 5 ):1.

MU Y , LIU W , LIU X , et al . Stochastic gradient made stable:a manifold propagation approach for large-scale optimization [J ] . IEEE Transactions on Knowledge ＆ Data Engineering , 2015 , 29 ( 2 ): 458 - 471 .

ZINKEVICH M , WEIMER M , SMOLA A J , et al . Parallelized stochastic gradient descent [C ] // The Conference on Neural Information Processing Systems . 2011 : 2595 - 2603 .

陈振宏 , 兰艳艳 , 郭嘉丰 , 等 . 基于差异合并的分布式随机梯度下降算法 [J ] . 计算机学报 , 2015 , 38 ( 10 ): 2054 - 2063 .

CHEN Z H , LAN Y Y , GUO J F , et al . Distributed stochastic gradient descent with discriminative aggregating [J ] . Chinese Journal of Computers , 2015 , 38 ( 10 ): 2054 - 2063 .

ZHAO H , CANNY J F . Communication-efficient distributed stochastic gradient descent with butterfly mixing [D ] . Berkeley,USA:University of California , 2012 .

SONG S , CHAUDHURI K , SARWATE A D . Stochastic gradient descent with differentially private updates [C ] // Global conference on Signal and Information Processing (GlobalSIP) . 2013 : 245 - 248 .

BASSILY R , THAKURTA A . Private empirical risk minimization:Efficient algorithms and tight error bounds [C ] // 2014 IEEE 55th Annual Symposium on Foundations of Computer Science (FOCS) . 2014 : 464 - 473 .

DWORK C , MCSHERRY F , NISSIM K . Calibrating noise to sensitivity in private data analysis [J ] . The VLDB Endowment , 2006 , 7 ( 8 ): 637 - 648 .

DWORK C , ROTH A . The Algorithmic foundations of differential privacy [M ] . Now Publishers Inc , 2014 .

CHAUDHURI K , MONTELEONI C , SARWATE A D . Differentially private empirical risk minimization [J ] . Journal of Machine Learning Research , 2009 , 12 ( 2 ): 1069 - 1109 .

何贤芒 , 王晓阳 , 陈华辉 , 等 . 差分隐私保护参数ε的选取研究 [J ] . 通信学报 , 2015 , 36 ( 12 ): 124 - 130 .

HE X M , WANG X Y , CHEN H H , et al . Study on choosing the parameter ε in differential privacy [J ] . Journal on Communications , 2015 , 36 ( 12 ): 124 - 130 .

MCSHERRY F D . Privacy integrated queries:an extensible platform for privacy-preserving data analysis [J ] . Communication of the ACM , 2010 , 53 ( 9 ): 89 - 97 .

浏览量

1508

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

MapReduce框架下支持差分隐私保护的k-means聚类方法

基于机器学习的加密流量分类研究综述

面向代码重用检测的程序语义分析模型

工业互联网流量分析技术综述

同态明文-密文矩阵运算及其应用