基于MapReduce的HDFS数据窃取随机检测算法

高元照; 李炳龙; 陈性元

doi:10.11959/j.issn.1000-436x.2018222

您当前的位置：

首页 >

文章列表页 >

基于MapReduce的HDFS数据窃取随机检测算法

学术论文 | 更新时间：2024-06-05

- 基于MapReduce的HDFS数据窃取随机检测算法
- Stochastic algorithm for HDFS data theft detection based on MapReduce
- 通信学报 2018年39卷第10期页码：11-21
- 作者机构：
  
  1. 信息工程大学三院，河南郑州 450001
  2. 密码科学技术国家重点实验室，北京 100094
- 作者简介：
  
  [ "高元照（1992-），男，河北衡水人，信息工程大学博士生，主要研究方向为云计算取证、大数据安全。" ]
  [ "李炳龙（1974-），男，河南卫辉人，博士，信息工程大学副教授、硕士生导师，主要研究方向为数字取证。" ]
  [ "陈性元（1963-），男，安徽无为人，博士，信息工程大学教授、博士生导师，主要研究方向为网络与信息安全。" ]
- 基金信息：
  
  国家高技术研究发展计划（“863”计划）基金资助项目(2015AA016006);国家自然科学基金资助项目(61702550)
- DOI：10.11959/j.issn.1000-436x.2018222
  中图分类号： TP311.1
- 网络出版日期：2018-10，
  
  纸质出版日期：2018-10-25
- 稿件说明：
移动端阅览
高元照, 李炳龙, 陈性元. 基于MapReduce的HDFS数据窃取随机检测算法[J]. 通信学报, 2018,39(10):11-21.

Yuanzhao GAO, Binglong LI, Xingyuan CHEN. Stochastic algorithm for HDFS data theft detection based on MapReduce[J]. Journal on communications, 2018, 39(10): 11-21.
高元照, 李炳龙, 陈性元. 基于MapReduce的HDFS数据窃取随机检测算法[J]. 通信学报, 2018,39(10):11-21. DOI： 10.11959/j.issn.1000-436x.2018222.

Yuanzhao GAO, Binglong LI, Xingyuan CHEN. Stochastic algorithm for HDFS data theft detection based on MapReduce[J]. Journal on communications, 2018, 39(10): 11-21. DOI： 10.11959/j.issn.1000-436x.2018222.

摘要

为了解决分布式云计算存储的数据窃取检测中，出现数据量大、内部窃取难以检测的问题，以 hadoop分布式文件系统（HDFS

hadoop distributed file system）为检测对象，提出了一种基于MapReduce的数据窃取随机检测算法。分析HDFS文件夹复制产生的MAC时间戳特性，确立复制行为的检测与度量方法，确保能够检测包括内部窃取的所有窃取模式。设计适合于 MapReduce 任意的任务划分，同时记录 HDFS 层次关系的输入数据集，实现海量时间戳数据的高效分析。实验结果表明，该算法能够通过分段检测策略很好地控制漏检率和误检文件夹数量，并且具有较高的执行效率和良好的可扩展性。

Abstract

To address the problems of big data efficient analysis and insider theft detection in the data theft detection of distributed cloud computing storage

taking HDFS (hadoop distributed file system) as a case study

a stochastic algorithm for HDFS data theft detection based on MapReduce was proposed.By analyzing the MAC timestamp features of HDFS generated by folder replication

the replication behavior’s detection and measurement method was established to detect all data theft modes including insider theft.The data set which is suitable for MapReduce task partition and maintains the HDFS hierarchy was designed to achieve efficient analysis of large-volume timestamps.The experimental results show that the missed rate and the number of mislabeled folders could be kept at a low level by adopting segment detection strategy.The algorithm was proved to be efficient and had good scalability under the MapReduce framework.

关键词

Keywords

references

中国信息通信研究院 . 云计算白皮书 [R ] . 北京:中国信息通信研究院 , 2016 .

China Academy of Information and Communication Technology . >云计算白皮书 > [R ] . White Papers of Cloud Computing [R ] . Beijing:China Academy of Information and Communication Technology , 2016 .

张玉清 , 王晓菲 , 刘雪峰 , 等 . 云计算环境安全综述 [J ] . 软件学报 , 2016 , 27 ( 6 ): 1328 - 1348 .

ZHANG Y Q , WANG X F , LIU X F , et al . Survey on cloud computing security [J ] . Journal of Software , 2016 , 27 ( 6 ): 1328 - 1348 .

CHANG V , RAMACHANDRAN M . Towards achieving data security with the cloud computing adoption framework [J ] . IEEE Trans Services Computing , 2016 , 9 ( 1 ): 138 - 151 .

MARTINI B , CHOO K K R . Cloud storage forensics:owncloud as a case study [J ] . Digital Investigation , 2013 , 10 ( 4 ): 287 - 299 .

LI Y , GAI K , QIU L , et al . Intelligent cryptography approach for secure distributed big data storage in cloud computing [J ] . Information Sciences , 2017 , 387 : 103 - 115 .

ALVA A , CALEFF O , ELKINS G , et al . The Notorious nine:cloud computing top threats in 2013 [R ] . Cloud Security Alliance , 2013 .

BROOK J M , FIELD S , SHACKLEFORD D , et al . The treacherous 12 - cloud computing top threats in 2016 [R ] . Seattle:Cloud Security Alliance , 2016 .

中国云计算安全政策与法律工作组 . 中国云计算安全政策与法律蓝皮书（2016） [R ] . 上海:中国云计算安全政策与法律工作组 , 2016 .

Cloud Computing Security Policies and Laws Group . Cloud computing security policies and laws blue book(2016) [R ] . Shanghai:Cloud Computing Security Policies and Laws Group , 2016 .

STOLFO S J , SALEM M B , KEROMYTIS A D . Fog computing:mitigating insider data theft attacks in the cloud [C ] // IEEE Symposium on Security and Privacy Workshops . 2012 : 125 - 128 .

SRIRAM M , PATEL V , HARISHMA D , et al . A hybrid protocol to secure the cloud from insider threats [C ] // IEEE International Conference on Cloud Computing in Emerging Markets . 2014 : 1 - 5 .

NIKOLAI J , WANG Y . A system for detecting malicious insider data theft in IaaS cloud environments [C ] // 2016 IEEE Global Communications Conference (GLOBECOM) . 2016 : 1 - 6 .

SUBASHINI S , KAVITHA V . A survey on security issues in service delivery models of cloud computing [J ] . Journal of Network ＆ Computer Applications , 2011 , 34 ( 1 ): 1 - 11 .

ROCHA F , CORREIA M . Lucy in the sky without diamonds:Stealing confidential data in the cloud [C ] // IEEE/IFIP International Conference on Dependable Systems and Networks Workshops . 2011 : 129 - 134 .

PITROPAKIS N , LYVAS C , LAMBRINOUDAKIS C . The greater the power,the more dangerous the abuse:facing malicious insiders in the cloud [J ] . Cloud Computing 2017 , 2017 : 156 - 161 .

MARTINI B , CHOO K K R . Distributed filesystem forensics:XtreemFS as a case study [J ] . Digital Investigation , 2014 , 11 ( 4 ): 295 - 313 .

GRIER J . Detecting data theft using stochastic forensics [J ] . Digital Investigation , 2011 , 8 ( 8 ): 71 - 77 .

PATEL P C , SINGH U . A novel classification model for data theft detection using advanced pattern mining [J ] . Digital Investigation , 2013 , 10 ( 4 ): 385 - 397 .

SHVACHKO K , KUANG H , RADIA S , et al . The hadoop distributed file system [C ] // 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST) . 2010 : 1 - 10 .

GAO Y , LI B . A forensic method of efficient file extraction in HDFS based on three-level mapping [J ] . Wuhan University Journal of Natural Sciences , 2017 , 22 ( 2 ): 114 - 126 .

WHITE T . hadoop:The definitive guide(3th Edition) [M ] . Sebastopol : O'Reilly Media,IncPress , 2015 .

BARHAM P , DRAGOVIC B , FRASER K , et al . Xen and the art of virtualization [C ] // ACM SIGOPS Operating Systems Review . 2003 : 164 - 177 .

MACKAY E B , CHALLENOR P G , BAHAJ A S . A comparison of estimators for the generalised Pareto distribution [J ] . Ocean Engineering , 2011 , 38 ( 11 ): 1338 - 1346 .

浏览量

1156

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

MapReduce框架下支持差分隐私保护的随机梯度下降算法

基于节点属性与正文内容的海量Web信息抽取方法

MapReduce框架下支持差分隐私保护的k-means聚类方法

温度感知的MapReduce节能任务调度策略

面向大数据的数据服务中心设计与应用研究