High-performance and high-concurrency encryption scheme for Hadoop platform

Wei JIN; Mingjie YU; Fenghua LI; Zhengkun YANG; Kui GENG

doi:10.11959/j.issn.1000-436x.2019224

您当前的位置：

首页 >

文章列表页 >

High-performance and high-concurrency encryption scheme for Hadoop platform

Papers | 更新时间：2024-06-05

- High-performance and high-concurrency encryption scheme for Hadoop platform
- Journal on Communications Vol. 40, Issue 12, Pages: 29-40(2019)
- 作者机构：
  
  1. 中国科学院信息工程研究所，北京100093
  2. 中国科学院大学网络空间安全学院，北京 100049
  3. 中国科学技术大学网络空间安全学院，安徽合肥 230027
- 作者简介：
- 基金信息：
  
  The National Key Research Project and Development Program of China(2017YFB0802705);The Strategic Priority Research Program of the Chinese Academy of Sciences(XDC02040400);The National Natural Science Foundation of China(U1836203)
- DOI：10.11959/j.issn.1000-436x.2019224
  CLC： TP302
- Online First：2019-12，
  
  Published：25 December 2019
- 稿件说明：
移动端阅览
Wei JIN, Mingjie YU, Fenghua LI, et al. High-performance and high-concurrency encryption scheme for Hadoop platform[J]. Journal on Communications, 2019, 40(12): 29-40.
DOI：

Wei JIN, Mingjie YU, Fenghua LI, et al. High-performance and high-concurrency encryption scheme for Hadoop platform[J]. Journal on Communications, 2019, 40(12): 29-40. DOI： 10.11959/j.issn.1000-436x.2019224.

摘要

针对Hadoop平台静态存储加密方案的加密算法单一、密钥管理复杂、加解密性能低，不能高效保护Hadoop数据安全的问题，提出了一套基于商用密码算法的 Hadoop 高性能加密与密钥管理方案。首先，提出基于国产商密算法的 Hadoop 平台三层密钥管理体系，优化二级密钥的组织结构；其次，提出异步流水模式的高并发加解密方法，替代 Hadoop 原有的串行加解密流程，并通过密文排序确保多加密线程的密文同步。实验结果表明，所提方案的密钥存取效率、文件读写速度快于原有Hadoop方案，所提方法可有效提升Hadoop平台密钥存取与加解密速度。

Abstract

To address the problem that as preventing data leakage on Hadoop platform

the existing encryption schemes suffer from several problems (e.g.

single encryption algorithm

complicated key management

low encryption performance) and they cannot protect data stored in Hadoop effectively

a high-performance encryption and key management scheme for Hadoop was proposed.Firstly

a three-level key management system was extended with the domestic commercial cipher algorithm.Then

a new data structure for encryption zone key to reduce time consumption was designed.Finally

the computing process of data stream in parallel was scheduled.The experimental results show that compared with the existing Hadoop schemes

the proposed scheme can improve the efficiency of key management

and can speed up file encryption.

关键词

Keywords

references

陈丽 , 黄晋 , 王锐 . Hadoop大数据平台安全问题和解决方案的综述 [J ] . 计算机系统应用 , 2018 , 27 ( 1 ): 1 - 9 .

CHEN L , HUANG J , WANG R . Overview on security issues and solutions of Hadoop big data platform [J ] . Computer Systems ＆ Applications , 2018 , 27 ( 1 ): 1 - 9 .

梁胜昔 , 秦军 , 宋蕾 . HDFS 混合加密保护方案的设计 [J ] . 计算机时代 , 2014 ( 7 ): 17 - 19 .

LIANG S X , QIN J , SONG L . Design for protection scheme of hybrid encryption in HDFS [J ] . Computer Era , 2014 ( 7 ): 17 - 19 .

DÍAZ A F , BLOKHIN I , ORTEGA J . Secure data access in Hadoop using elliptic curve cryptography [C ] // International Conference on Algorithms and Architectures for Parallel Processing . 2016 : 136 - 145 .

WANG F , KOHLER M , SCHAAD A . Initial encryption of large searchable data sets using Hadoop [C ] // The 20th ACM Symposium on Access Control Models and Technologies . SACMAT , 2015 : 165 - 168 .

SONG Y , SHIN Y S , JANG M . Design and implementation of HDFS data encryption scheme using ARIA algorithm on Hadoop [C ] // 2017 IEEE International Conference on Big Data and Smart Computing . IEEE , 2017 : 84 - 90 .

LIN H Y , SHEN S T , TZENG W G , et al . Toward data confidential ity via integrating hybrid encryption schemes and Hadoop distributed file system [C ] // 2012 IEEE 26th International Conference on Advanced Information Networking and Applications (AINA) . IEEE , 2012 : 740 - 747 .

BHATOTIA P , RODRIGUES R , VERMA A . Shredder:GPU-accelerated incremental storage and computation [C ] // The 10th USENIX conference on File and Storage Technologies . 2012 .

COHEN J , ACHARYA S . Towards a trusted Hadoop storage platform:design considerations of an AES based encryption scheme with TPM rooted key protections [C ] // The 2013 IEEE 10th International Conference on Ubiquitous Intelligence ＆ Computing and 2013 IEEE 10th International Conference on Autonomic ＆ Trusted Computing . IEEE , 2013 : 444 - 451 .

ZHOU W , XU Y , WANG G . Research on key management for multi-privileged group communications [C ] // IEEE Trustcom/BigDataSE/ISPA . IEEE Computer Society , 2015 : 151 - 158 .

KANDAH F I , NICHOLS O , YANG L . Efficient key management for big data gathering in dynamic sensor networks [C ] // International Conference on Computing . IEEE , 2017 : 667 - 671 .

ALBAKRI A , MADDUMALA M , HARN L . Hierarchical polynomial-based key management scheme in fog computing [C ] // IEEE TrustCom/BigDataSE . IEEE , 2018 : 1593 - 1597 .

Views

2410

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Efficient unpaired data validation and aggregation protocol in industrial Internet of things

Blockchain-based distributed EHR fine-grained traceability scheme

Secure deduplication and integrity audit system based on convergent encryption for cloud storage

Dynamic accumulators-based authenticated group key management scheme for heterogeneous wireless sensor network

Ternary polynomial based key management scheme for wireless sensor network

Related Author

FENG Tao

MA Rong

Ximeng LIU

Jianfeng MA

Yuanping SI

Zuobin YING

Wei-jia DING

Bo-yu KUANG

Related Institution

School of Computer and Communication, Lanzhou University of Technology

School of Mathematics and Computer Science, Fuzhou University

School of Computer Science and Technology, Anhui University

Institutes of Physical and Information Technology, Anhui University

School of Network and Information Security, Xidian University

AI问答

⁰