Hierarchical scalable storage architecture for massive electronic bill

Fenghua LI; Dingyan LI; Wei JIN; Zhu WANG; Yunchuan GUO; Kui GENG

doi:10.11959/j.issn.1000-436x.2019104

您当前的位置：

首页 >

文章列表页 >

Hierarchical scalable storage architecture for massive electronic bill

Papers | 更新时间：2024-06-05

- Hierarchical scalable storage architecture for massive electronic bill
- Journal on Communications Vol. 40, Issue 5, Pages: 79-87(2019)
- 作者机构：
  
  1. 中国科学院信息工程研究所，北京 100093
  2. 中国科学院大学网络空间安全学院，北京 100049
- 作者简介：
- 基金信息：
  
  The National Key Research and Development Program of China(2017YFB0802702);The National Natural Science Foundation of China(61672515)
- DOI：10.11959/j.issn.1000-436x.2019104
  CLC： TP311
- Online First：2019-05，
  
  Published：25 May 2019
- 稿件说明：
移动端阅览
Fenghua LI, Dingyan LI, Wei JIN, et al. Hierarchical scalable storage architecture for massive electronic bill[J]. Journal on Communications, 2019, 40(5): 79-87.
DOI：

Fenghua LI, Dingyan LI, Wei JIN, et al. Hierarchical scalable storage architecture for massive electronic bill[J]. Journal on Communications, 2019, 40(5): 79-87. DOI： 10.11959/j.issn.1000-436x.2019104.

摘要

电子商务等网络服务的兴起极大地促进了电子凭据业务的发展，传统数据存储方案无法满足海量电子凭据数据的快速访问需求。针对上述问题，提出了一种面向海量电子凭据的分层可扩展存储架构，结合hash取模算法和一致性hash算法实现快速的数据定位，设计了基于hash取模算法的横向扩展方案，减少了节点增删时迁移的数据量。此外，设计并实现了基于热数据的缓存方案和基于访问时延的负载均衡方案，进一步提升数据访问的速度。最后通过实验证明了所提架构与方案的有效性。

Abstract

The rise of e-commerce and other network services have greatly promoted the development of electronic bill service

while traditional data storage schemes can no longer satisfy the rapid access requirements of massive electronic bill data.To solve these problems

a hierarchical scalable storage architecture for massive electronic bill was proposed

which combined hash modular algorithm and consistent hash algorithm

and supported fast data locating.A horizontal expansion scheme based on hash modular algorithm was designed to reduce the amount of data that needed to migrate when adding or deleting data nodes.Besides

a data caching scheme based on hot data and a load balancing scheme based on access delay were designed and implemented

further improving the speed of data access.Finally

the experiments prove the effectiveness of proposed architecture and schemes.

关键词

Keywords

references

MAKRIS A , TSERPES K , ANDRONIKOU V , et al . A classification of NoSQL data stores based on key design characteristics [J ] . Procedia Computer Science , 2016 , 97 : 94 - 103 .

CORBELLINI A , MATEOS C , ZUNINO A , et al . Persisting big data:the NoSQL landscape [J ] . Information Systems , 2017 , 63 : 1 - 23 .

SIDDIQA A , KARIM A , GANI A . Big data storage technologies:a survey [J ] . Frontiers of Information Technology and Electronic Engineering , 2017 , 18 ( 8 ): 1040 - 1070 .

DECANDIA G , HASTORUN D , JAMPANI M , et al . Dynamo:Amazon’s highly available key-value store [J ] . ACM SIGOPS Operating Systems Review , 2007 , 41 ( 6 ): 205 - 220 .

马友忠 , 孟小峰 . 云数据管理索引技术研究 [J ] . 软件学报 , 2015 , 26 ( 1 ): 145 - 166 .

MA Y Z , MENG X F . Research on indexing for cloud data management [J ] . Journal of Software , 2015 , 26 ( 1 ): 145 - 166 .

VICTOR Z , DIVYAKANT A , AMR E A . Caching at the web scale [J ] . Proceedings of the VLDB Endowment , 2017 ( 10 ): 2002 - 2005 .

ASAF C , ASSAF E , MOHAMMAD A , et al . Cliffhanger:scaling performance cliffs in web memory caches [C ] // 13th USENIX Symposium on Networked Systems Design and Implementation . USENIX , 2016 : 379 - 392 .

JIANG Y . A survey of task allocation and load balancing in distributed systems [J ] . IEEE Transactions on Parallel and Distributed Systems , 2016 , 27 ( 2 ): 585 - 599 .

LIU J , PACITTI E , VALDURIEZ P . A survey of scheduling frameworks in big data systems [J ] . International Journal of Cloud Computing , 2018 ( 7 ): 103 - 128 .

AKRAM E , LARBI H , ABDERRAHIM M . The main characteristics of five distributed file systems required for big data:a comparative study [J ] . Advances in Science,Technology and Engineering Systems Journal , 2017 , 2 ( 4 ): 78 - 91 .

邹立达 , 李庆忠 , 孔兰菊 , 等 . 基于Chord的多租户索引机制研究 [J ] . 计算机学报 , 2016 , 39 ( 2 ): 270 - 285 .

ZOU L D , LI Q Z , KONG L J , et al . Indexing mechanism of multi-tenant database based on Chord [J ] . Chinese Journal of Computers , 2016 , 39 ( 2 ): 270 - 285 .

QAISAR J , AYESHA Z , MUHAMMAD A , et al . Cache memory:an analysis on replacement algorithms and optimization techniques [J ] . Mehran University Research Journal of Engineering and Technology , 2017 , 36 ( 4 ): 831 - 840 .

ZAIDENBERG N , GAVISH L , MEIR Y . New caching algorithms performance evaluation [C ] // International Symposium on Performance Evaluation of Computer and Telecommunication Systems . 2015 : 1 - 7 .

CIDON A , EISENMAN A , ALIZADEH M , et al . Dynacache:dynamic cloud caching [C ] // USENIX Conference on Hot Topics in Cloud Computing . USENIX , 2015 .

葛微 , 罗圣美 , 周文辉 , 等 . HiBase:一种基于分层式索引的高效HBase查询技术与系统 [J ] . 计算机学报 , 2016 ( 1 ): 140 - 153 .

GE W , LUO S M , ZHOU W H , et al . HiBase:a hierarchical indexing mechanism and system for efficient HBase query [J ] . Chinese Journal of Computers , 2016 ( 1 ): 140 - 153 .

RATHORE N . Performance of hybrid load balancing algorithm in distributed web server system [J ] . Wireless Personal Communications , 2018 ( 101 ): 1233 - 1246 .

REN G , JUEBO W . Dynamic load balancing strategy for cloud computing with ant colony optimization [J ] . Future Internet , 2015 , 7 ( 4 ): 465 - 483 .

YANG Z X , ZHANG S Y , JI X Y , et al . Research on cloud service quality control implementation based on improved load balance algorithm [J ] . Journal of Computational Methods in Sciences and Engineering , 2018 , 18 ( 3 ): 793 - 800 .

HERBST N , AMIN A , ANDRZEJAK A , et al . Online workload forecasting [C ] // Self-Aware Computing Systems . 2017 : 529 - 553 .

邢屹 . 大规模键值分布式存储系统的设计与实现 [D ] . 成都:电子科技大学 , 2013 .

XING Y . The design and implementation of a large-scale key-value distributed storage system [D ] . Chengdu:University of Electronic Science and Technology of China , 2013 .

佘楚玉 , 温武少 , 肖扬 , 等 . 一种自适应文件系统元数据服务负载均衡策略 [J ] . 软件学报 , 2017 , 28 ( 8 ): 1952 - 1967 .

SHE C Y , WEN W S , XIAO Y , et al . Adaptive load balancing strategy for file-system metadata service [J ] . Journal of Software , 2017 , 28 ( 8 ): 1952 - 1967 .

JOANNA B , MACIEJ D . Comparing load-balancing algorithms for mapreduce under zipfian data skews [J ] . Parallel Computing , 2018 , 72 : 14 - 28 .

Views

863

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Multi-indicator intention-driven adaptive optimization strategy for UAV computation offloading and trajectory planning

Traffic prediction and load balancing based on in-band network telemetry data

Coordinated UAV-UGV trajectory planning based on load balancing in IoT data collection

MgdFlow: multi-granularity data flow management algorithm in microgrid scenario

Neighbor satellite load based low orbit satellite distributed routing algorithm

Related Author

Zhang Zhizhong

Guo Lei

Liu Yan

Ning Zhaolong

Huang Xinliang

Lin Peng

GUO Lei

LIU Wei

Related Institution

School of Computer Science and Engineering, Northeastern University

School of Communication and Information Engineering, Chongqing University of Posts and Telecommunications

School of Electronic and Information Engineering, Nanjing University of Information Science and Technology

Key Laboratory of Intelligent Support Technology for Complex Environments, Ministry of Education, Nanjing University of Information Science and Technology

Key Laboratory of Big Data Intelligent Computing, Chongqing University of Posts and Telecommunications

AI问答

⁰