Stochastic algorithm for HDFS data theft detection based on MapReduce

Yuanzhao GAO; Binglong LI; Xingyuan CHEN

doi:10.11959/j.issn.1000-436x.2018222

您当前的位置：

首页 >

文章列表页 >

Stochastic algorithm for HDFS data theft detection based on MapReduce

Papers | 更新时间：2024-06-05

- Stochastic algorithm for HDFS data theft detection based on MapReduce
- Journal on Communications Vol. 39, Issue 10, Pages: 11-21(2018)
- 作者机构：
  
  1. 信息工程大学三院，河南郑州 450001
  2. 密码科学技术国家重点实验室，北京 100094
- 作者简介：
- 基金信息：
  
  The National High Technology Research and Development Program of China (863 Program)(2015AA016006);The National Natural Science Foundation of China(61702550)
- DOI：10.11959/j.issn.1000-436x.2018222
  CLC： TP311.1
- Online First：2018-10，
  
  Published：25 October 2018
- 稿件说明：
移动端阅览
Yuanzhao GAO, Binglong LI, Xingyuan CHEN. Stochastic algorithm for HDFS data theft detection based on MapReduce[J]. Journal on Communications, 2018, 39(10): 11-21.
DOI：

Yuanzhao GAO, Binglong LI, Xingyuan CHEN. Stochastic algorithm for HDFS data theft detection based on MapReduce[J]. Journal on Communications, 2018, 39(10): 11-21. DOI： 10.11959/j.issn.1000-436x.2018222.

摘要

为了解决分布式云计算存储的数据窃取检测中，出现数据量大、内部窃取难以检测的问题，以 hadoop分布式文件系统（HDFS

hadoop distributed file system）为检测对象，提出了一种基于MapReduce的数据窃取随机检测算法。分析HDFS文件夹复制产生的MAC时间戳特性，确立复制行为的检测与度量方法，确保能够检测包括内部窃取的所有窃取模式。设计适合于 MapReduce 任意的任务划分，同时记录 HDFS 层次关系的输入数据集，实现海量时间戳数据的高效分析。实验结果表明，该算法能够通过分段检测策略很好地控制漏检率和误检文件夹数量，并且具有较高的执行效率和良好的可扩展性。

Abstract

To address the problems of big data efficient analysis and insider theft detection in the data theft detection of distributed cloud computing storage

taking HDFS (hadoop distributed file system) as a case study

a stochastic algorithm for HDFS data theft detection based on MapReduce was proposed.By analyzing the MAC timestamp features of HDFS generated by folder replication

the replication behavior’s detection and measurement method was established to detect all data theft modes including insider theft.The data set which is suitable for MapReduce task partition and maintains the HDFS hierarchy was designed to achieve efficient analysis of large-volume timestamps.The experimental results show that the missed rate and the number of mislabeled folders could be kept at a low level by adopting segment detection strategy.The algorithm was proved to be efficient and had good scalability under the MapReduce framework.

关键词

Keywords

references

中国信息通信研究院 . 云计算白皮书 [R ] . 北京:中国信息通信研究院 , 2016 .

China Academy of Information and Communication Technology . >云计算白皮书 > [R ] . White Papers of Cloud Computing [R ] . Beijing:China Academy of Information and Communication Technology , 2016 .

张玉清 , 王晓菲 , 刘雪峰 , 等 . 云计算环境安全综述 [J ] . 软件学报 , 2016 , 27 ( 6 ): 1328 - 1348 .

ZHANG Y Q , WANG X F , LIU X F , et al . Survey on cloud computing security [J ] . Journal of Software , 2016 , 27 ( 6 ): 1328 - 1348 .

CHANG V , RAMACHANDRAN M . Towards achieving data security with the cloud computing adoption framework [J ] . IEEE Trans Services Computing , 2016 , 9 ( 1 ): 138 - 151 .

MARTINI B , CHOO K K R . Cloud storage forensics:owncloud as a case study [J ] . Digital Investigation , 2013 , 10 ( 4 ): 287 - 299 .

LI Y , GAI K , QIU L , et al . Intelligent cryptography approach for secure distributed big data storage in cloud computing [J ] . Information Sciences , 2017 , 387 : 103 - 115 .

ALVA A , CALEFF O , ELKINS G , et al . The Notorious nine:cloud computing top threats in 2013 [R ] . Cloud Security Alliance , 2013 .

BROOK J M , FIELD S , SHACKLEFORD D , et al . The treacherous 12 - cloud computing top threats in 2016 [R ] . Seattle:Cloud Security Alliance , 2016 .

中国云计算安全政策与法律工作组 . 中国云计算安全政策与法律蓝皮书（2016） [R ] . 上海:中国云计算安全政策与法律工作组 , 2016 .

Cloud Computing Security Policies and Laws Group . Cloud computing security policies and laws blue book(2016) [R ] . Shanghai:Cloud Computing Security Policies and Laws Group , 2016 .

STOLFO S J , SALEM M B , KEROMYTIS A D . Fog computing:mitigating insider data theft attacks in the cloud [C ] // IEEE Symposium on Security and Privacy Workshops . 2012 : 125 - 128 .

SRIRAM M , PATEL V , HARISHMA D , et al . A hybrid protocol to secure the cloud from insider threats [C ] // IEEE International Conference on Cloud Computing in Emerging Markets . 2014 : 1 - 5 .

NIKOLAI J , WANG Y . A system for detecting malicious insider data theft in IaaS cloud environments [C ] // 2016 IEEE Global Communications Conference (GLOBECOM) . 2016 : 1 - 6 .

SUBASHINI S , KAVITHA V . A survey on security issues in service delivery models of cloud computing [J ] . Journal of Network ＆ Computer Applications , 2011 , 34 ( 1 ): 1 - 11 .

ROCHA F , CORREIA M . Lucy in the sky without diamonds:Stealing confidential data in the cloud [C ] // IEEE/IFIP International Conference on Dependable Systems and Networks Workshops . 2011 : 129 - 134 .

PITROPAKIS N , LYVAS C , LAMBRINOUDAKIS C . The greater the power,the more dangerous the abuse:facing malicious insiders in the cloud [J ] . Cloud Computing 2017 , 2017 : 156 - 161 .

MARTINI B , CHOO K K R . Distributed filesystem forensics:XtreemFS as a case study [J ] . Digital Investigation , 2014 , 11 ( 4 ): 295 - 313 .

GRIER J . Detecting data theft using stochastic forensics [J ] . Digital Investigation , 2011 , 8 ( 8 ): 71 - 77 .

PATEL P C , SINGH U . A novel classification model for data theft detection using advanced pattern mining [J ] . Digital Investigation , 2013 , 10 ( 4 ): 385 - 397 .

SHVACHKO K , KUANG H , RADIA S , et al . The hadoop distributed file system [C ] // 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST) . 2010 : 1 - 10 .

GAO Y , LI B . A forensic method of efficient file extraction in HDFS based on three-level mapping [J ] . Wuhan University Journal of Natural Sciences , 2017 , 22 ( 2 ): 114 - 126 .

WHITE T . hadoop:The definitive guide(3th Edition) [M ] . Sebastopol : O'Reilly Media,IncPress , 2015 .

BARHAM P , DRAGOVIC B , FRASER K , et al . Xen and the art of virtualization [C ] // ACM SIGOPS Operating Systems Review . 2003 : 164 - 177 .

MACKAY E B , CHALLENOR P G , BAHAJ A S . A comparison of estimators for the generalised Pareto distribution [J ] . Ocean Engineering , 2011 , 38 ( 11 ): 1338 - 1346 .

Views

1985

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Stochastic gradient descent algorithm preserving differential privacy in MapReduce framework

Information extraction from massive Web pages based on node property and text content

k-means clustering method preserving differential privacy in MapReduce framework

Temperature aware energy-efficient task scheduling strategies for mapreduce

Design and application research on data service platform for big data

Related Author

Xiaoping WU

Yu FU

Yihan YU

Hai-yan WANG

Pan CAO

Yan CHEN

Xiao-ping WU

Hong-cheng LI

Related Institution

Department of Information Security, Naval University of Engineering

School of Computer Science and Technology, Nanjing University of Posts and Telecommunications

Jiangsu High Technology Research Key Laboratory for Wireless Sensor Networks

No.61062 Troops of PLA

College of Statistics and Information,Xinjiang University of Finance and Economics

AI问答

⁰