Review on autoencoder and its application

Jie LAI; Xiaodan WANG; Qian XIANG; Yafei SONG; Wen QUAN

doi:10.11959/j.issn.1000-436x.2021160

您当前的位置：

首页 >

文章列表页 >

Review on autoencoder and its application

Comprehensive Reviews | 更新时间：2024-06-05

- Review on autoencoder and its application
- Journal on Communications Vol. 42, Issue 9, Pages: 218-230(2021)
- 作者机构：
  
  1. 空军工程大学防空反导学院，陕西西安 710051
  2. 空军工程大学空管领航学院，陕西西安 710051
- 作者简介：
- 基金信息：
  
  The National Natural Science Foundation of China(61876189);The National Natural Science Foundation of China(61806219);The National Natural Science Foundation of China(61703426);The Natural Science Basic Research Plan in Shaanxi Province(2021JM—226)
- DOI：10.11959/j.issn.1000-436x.2021160
  CLC： TP183
- Online First：2021-09，
  
  Published：25 September 2021
- 稿件说明：
移动端阅览
Jie LAI, Xiaodan WANG, Qian XIANG, et al. Review on autoencoder and its application[J]. Journal on Communications, 2021, 42(9): 218-230.
DOI：

Jie LAI, Xiaodan WANG, Qian XIANG, et al. Review on autoencoder and its application[J]. Journal on Communications, 2021, 42(9): 218-230. DOI： 10.11959/j.issn.1000-436x.2021160.

摘要

自编码器作为典型的深度无监督学习模型，能够从无标签样本中自动学习样本的有效抽象特征。近年来，自编码器受到广泛关注，已应用于目标识别、入侵检测、故障诊断等众多领域中。基于此，对自编码器的理论基础、改进技术、应用领域与研究方向进行了较全面的阐述与总结。首先，介绍了传统自编码器的网络结构与理论推导，分析了自编码器的算法流程，并与其他无监督学习算法进行了比较。然后，讨论了常用的自编码器改进算法，分析了其出发点、改进方式与优缺点。接着，介绍了自编码器在目标识别、入侵检测等具体领域的实际应用现状。最后，总结了现有自编码器及其改进算法存在的问题，并展望了自编码器的研究方向。

Abstract

As a typical deep unsupervised learning model

autoencoder can automatically learn effective abstract features from unlabeled samples.In recent years

autoencoder has been widely used in target recognition

intrusion detection

fault diagnosis and many other fields.Thus

the theoretical basis

improved methods

application fields and research directions of autoencoder were described and summarized comprehensively.At first

the network structure

theoretical derivation and algorithm flow of traditional autoencoder were introduced and analyzed

and the difference between autoencoder and other unsupervised learning algorithms was compared.Then

common improved autoencoders were discussed

and their innovation

improvement methods and relative merits were analyzed.Next

the practical application status of autoencoder in target recognition

intrusion detection and other fields were introduced.At last

the existing problems of autoencoder were summarized

and the possible research directions were prospected.

关键词

Keywords

references

HINTON G E , SALAKHUTDINOV R R . Reducing the dimensionality of data with neural networks [J ] . Science , 2006 , 313 ( 5786 ): 504 - 507 .

BENGIO Y , COURVILLE A , VINCENT P . Representation learning:a review and new perspectives [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2013 , 35 ( 8 ): 1798 - 1828 .

SCHOLKOPF B , PLATT J , HOFMANN T . Greedy layer-wise training of deep networks [J ] . Advances in Neural Information Processing Systems , 2007 , 19 : 153 - 160 .

袁非牛 , 章琳 , 史劲亭 , 等 . 自编码神经网络理论及应用综述 [J ] . 计算机学报 , 2019 , 42 ( 1 ): 203 - 230 .

YUAN F N , ZHANG L , SHI J T , et al . Theories and applications of auto-encoder neural networks:a literature survey [J ] . Chinese Journal of Computers , 2019 , 42 ( 1 ): 203 - 230 .

RUMELHART D E , HINTON G E , WILLIAMS R J . Learning representations by back-propagating errors [J ] . Nature , 1986 , 323 ( 6088 ): 533 - 536 .

BOURLARD H , KAMP Y . Auto-association by multilayer perceptrons and singular value decomposition [J ] . Biological Cybernetics , 1988 , 59 ( 4/5 ): 291 - 294 .

SALAKHUTDINOV R , HINTON G . Deep boltzmann machines [C ] // Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics . Clearwater Beach:MLRP , 2009 : 448 - 455 .

FISCHER A , IGEL C . Training restricted Boltzmann machines:an introduction [J ] . Pattern Recognition , 2014 , 47 ( 1 ): 25 - 39 .

GOODFELLOW I , POUGET-ABADIE J , MIRZA M , et al . Generative adversarial nets [C ] // Proceedings of the Advances in Neural Information Processing Systems . Massachusetts:MIT Press , 2014 : 2672 - 2680 .

NG A . Sparse autoencoder [J ] . CS294A Lecture Notes , 2011 , 72 ( 1 ): 1 - 19 .

VINCENT P , LAROCHELLE H , LAJOIE I , et al . Stacked denoising autoencoders:learning useful representations in a deep network with a local denoising criterion [J ] . Journal of Machine Learning Research , 2010 , 11 ( 12 ): 3371 - 3408 .

RIFAI S , VINCENT P , MULLER X , et al . Contractive auto-encoders:explicit invariance during feature extraction [C ] // Proceedings of the 28th International Conference on Machine Learning . Bellevue:Omnipress , 2011 : 833 - 840 .

KINGMA D P , WELLING M . Auto-encoding variational bayes [J ] . arXiv Preprint,arXiv:1312.6114 , 2013 .

MIAN P , JIE J , ZHU L , et al . Radar HRRP recognition based on discriminant deep autoencoders with small training data size [J ] . Electronics Letters , 2016 , 52 ( 20 ): 1725 - 1727 .

FENG B , CHEN B , LIU H W . Radar HRRP target recognition with deep networks [J ] . Pattern Recognition , 2017 , 61 : 379 - 393 .

LI Y C , MA R , JIAO R H . A hybrid malicious code detection method based on deep learning [J ] . International Journal of Security and Its Applications , 2015 , 9 ( 5 ): 205 - 216 .

JAVAID A , NIYAZ Q , SUN W Q , et al . A deep learning approach for network intrusion detection system [C ] // Proceedings of the 9th EAI International Conference on Bio-inspired Information and Communications Technologies . New York:ACM Press , 2016 : 21 - 26 .

ZHANG Y Y , GAO L , LI X Y , et al . A novel data-driven fault diagnosis method based on deep learning [C ] // International Conference on Data Mining and Big Data . Berlin:Springer , 2017 : 442 - 452 .

LU C , WANG Z Y , QIN W L , et al . Fault diagnosis of rotary machinery components using a stacked denoising autoencoder-based health state identification [J ] . Signal Processing , 2017 , 130 : 377 - 388 .

ZHANG J , YU J , TAO D C . Local deep-feature alignment for unsupervised dimension reduction [J ] . IEEE Transactions on Image Processing , 2018 , 27 ( 5 ): 2420 - 2432 .

WANG J L , HOU B , JIAO L C , et al . POL-SAR image classification based on modified stacked autoencoder network and data distribution [J ] . IEEE Transactions on Geoscience and Remote Sensing , 2020 , 58 ( 3 ): 1678 - 1695 .

LIU C , TANG L X , LIU J Y . A stacked autoencoder with sparse Bayesian regression for end-point prediction problems in steelmaking process [J ] . IEEE Transactions on Automation Science and Engineering , 2020 , 17 ( 2 ): 550 - 561 .

KOHONEN T , HONKELA T . Kohonen network [J ] . Scholarpedia , 2007 , 2 ( 1 ): 1568 .

胡铭菲 , 刘建伟 , 左信 . 深度生成模型综述 [J ] . 自动化学报 , 2020 , 41 : 1 - 35 .

HU M F , LIU J W , ZUO X . Survey on deep generative model [J ] . Acta Automatica Sinica , 2020 , 41 : 1 - 35 .

翟正利 , 梁振明 , 周炜 , 等 . 变分自编码器模型综述 [J ] . 计算机工程与应用 , 2019 , 55 ( 3 ): 1 - 9 .

ZHAI Z L , LIANG Z M , ZHOU W , et al . Research overview of variational auto-encoders models [J ] . Computer Engineering and Applications , 2019 , 55 ( 3 ): 1 - 9 .

MASCI J , MEIER U , CIREŞAN D , et al . Stacked convolutional auto-encoders for hierarchical feature extraction [C ] // Lecture Notes in Computer Science . Berlin:Springer , 2011 : 52 - 59 .

KASUN L L C , ZHOU H , HUANG G B , et al . Representational learning with extreme learning machine for big data [J ] . IEEE Intelligent Systems , 2013 , 28 ( 6 ): 31 - 34 .

XIE J , FANG Y , ZHU F , et al . Deepshape:deep learned shape descriptor for 3D shape matching and retrieval [C ] // 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway:IEEE Press , 2015 : 1275 - 1283 .

LI R , WANG X D , LEI L , et al . L 21 -norm based loss function and regularization extreme learning machine [J ] . IEEE Access , 2019 , 7 : 6575 - 6586 .

MAKHZANI A , SHLENS J , JAITLY N , et al . Adversarial autoencoders [J ] . arXiv Preprint,arXiv:1511.05644 , 2015 .

MAJUMDAR A . Graph structured autoencoder [J ] . Neural Networks , 2018 , 106 : 271 - 280 .

FAN F L , SHAN H M , KALRA M K , et al . Quadratic autoencoder (Q-AE) for low-dose CT denoising [J ] . IEEE Transactions on Medical Imaging , 2020 , 39 ( 6 ): 2035 - 2050 .

JOO W , LEE W , PARK S , et al . Dirichlet variational autoencoder [J ] . Pattern Recognition , 2020 , 107 : 107514 - 2050 .

CHEN S , WU M . Attention collaborative autoencoder for explicit recommender systems [J ] . Electronics , 2020 , 9 ( 10 ): 1716 .

YUAN X F , WANG Y L , YANG C H , et al . Stacked isomorphic autoencoder based soft analyzer and its application to sulfur recovery unit [J ] . Information Sciences , 2020 , 534 : 72 - 84 .

WANG X D , LI R , WANG J , et al . One-dimension hierarchical local receptive fields based extreme learning machine for radar target HRRP recognition [J ] . Neurocomputing , 2020 , 418 : 314 - 325 .

ZHAO F X , LIU Y X , HUO K , et al . Radar HRRP target recognition based on stacked autoencoder and extreme learning machine [J ] . Sensors , 2018 , 18 ( 2 ): 173 .

CHEN X Y , PENG X Y , LI J B , et al . Sparse autoencoder based manifold analyzer model of multi-angle target feature [J ] . IEEE Access , 2020 , 8 : 153250 - 153263 .

KANG M , JI K F , LENG X G , et al . Synthetic aperture radar target recognition with feature fusion based on a stacked autoencoder [J ] . Sensors , 2017 , 17 ( 12 ): 192 - 197 .

DENG S , DU L , LI C , et al . SAR automatic target recognition based on euclidean distance restricted autoencoder [J ] . IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 2017 , 10 ( 7 ): 3323 - 3333 .

DONG G G , LIAO G S , LIU H W , et al . A review of the autoencoder and its variants:a comparative perspective from target recognition in synthetic-aperture radar images [J ] . IEEE Geoscience and Remote Sensing Magazine , 2018 , 6 ( 3 ): 44 - 68 .

GUO J , WANG L , ZHU D Y , et al . Compact convolutional autoencoder for SAR target recognition [J ] . IET Radar,Sonar ＆ Navigation , 2020 , 14 ( 7 ): 967 - 972 .

MISHRA P , VARADHARAJAN V , TUPAKULA U , et al . A detailed investigation and analysis of using machine learning techniques for intrusion detection [J ] . IEEE Communications Surveys ＆ Tutorials , 2019 , 21 ( 1 ): 686 - 728 .

AL-QATF M , YU L S , AL-HABIB M , , et al . Deep learning approach combining sparse autoencoder with SVM for network intrusion detection [J ] . IEEE Access , 2018 , 6 : 52843 - 52856 .

CAO V L , NICOLAU M , MCDERMOTT J . Learning neural representations for network anomaly detection [J ] . IEEE Transactions on Cybernetics , 2019 , 49 ( 8 ): 3074 - 3087 .

CHOUHAN N , KHAN A , KHAN H U R . Network anomaly detection using channel boosted and residual learning based deep convolutional neural network [J ] . Applied Soft Computing , 2019 , 83 : 105612 .

LOPEZ-MARTIN M , CARRO B , SANCHEZ-ESGUEVILLAS A , . Variational data generative model for intrusion detection [J ] . Knowledge and Information Systems , 2019 , 60 ( 1 ): 569 - 590 .

IERACITANO C , ADEEL A , MORABITO F C , et al . A novel statistical analysis and autoencoder driven intelligent intrusion detection approach [J ] . Neurocomputing , 2020 , 387 : 51 - 62 .

TANG C F , LUKTARHAN N , ZHAO Y X . SAAE-DNN:deep learning method on intrusion detection [J ] . Symmetry , 2020 , 12 ( 10 ): 1695 .

樊红卫 , 张旭辉 , 曹现刚 , 等 . 智慧矿山背景下我国煤矿机械故障诊断研究现状与展望 [J ] . 振动与冲击 , 2020 , 39 ( 24 ): 194 - 204 .

FAN H W , ZHANG X H , CAO X G , et al . Research status and prospect of fault diagnosis of China’s coal mine machines under background of intelligent mine [J ] . Journal of Vibration and Shock , 2020 , 39 ( 24 ): 194 - 204 .

WEN L , GAO L , LI X Y . A new deep transfer learning based on sparse auto-encoder for fault diagnosis [J ] . IEEE Transactions on Systems,Man,and Cybernetics:Systems , 2019 , 49 ( 1 ): 136 - 144 .

LI K , XIONG M , LI F C , et al . A novel fault diagnosis algorithm for rotating machinery based on a sparsity and neighborhood preserving deep extreme learning machine [J ] . Neurocomputing , 2019 , 350 : 261 - 270 .

YU J B . A selective deep stacked denoising autoencoders ensemble with negative correlation learning for gearbox fault diagnosis [J ] . Computers in Industry , 2019 , 108 : 62 - 72 .

ZHAO D F , LIU S L , GU D , et al . Enhanced data-driven fault diagnosis for machines with small and unbalanced data based on variational auto-encoder [J ] . Measurement Science and Technology , 2020 , 31 ( 3 ): 035004 .

YU J B , ZHOU X K . One-dimensional residual convolutional autoencoder based feature learning for gearbox fault diagnosis [J ] . IEEE Transactions on Industrial Informatics , 2020 , 16 ( 10 ): 6347 - 6358 .

GAO W , QIAO S P , WAI R J , et al . A newly designed diagnostic method for mechanical faults of high-voltage circuit breakers via SSAE and IELM [J ] . IEEE Transactions on Instrumentation and Measurement , 2021 , 70 : 1 - 13 .

许卓斌 , 郑海山 , 潘竹虹 . 基于改进自编码器的文本分类算法 [J ] . 计算机科学 , 2018 , 45 ( 6 ): 208 - 210 , 240 .

XU Z B , ZHENG H S , PAN Z H . Improved autoencoder based classification algorithm for text [J ] . Computer Science , 2018 , 45 ( 6 ): 208 - 210 , 240 .

JIANG M Y , LIANG Y C , FENG X Y , et al . Text classification based on deep belief network and softmax regression [J ] . Neural Computing and Applications , 2018 , 29 ( 1 ): 61 - 70 .

LI Q , LI P F , MAO K Z , et al . Improving convolutional neural network for text classification by recursive data pruning [J ] . Neurocomputing , 2020 , 414 : 143 - 152 .

冀俊忠 , 庞皓明 , 杨翠翠 , 等 . 基于多隐层极限学习机的文本分类方法 [J ] . 北京工业大学学报 , 2019 , 45 ( 6 ): 534 - 545 .

JI J Z , PANG H M , YANG C C , et al . Text classification method based on multi-layer extreme learning machine [J ] . Journal of Beijing University of Technology , 2019 , 45 ( 6 ): 534 - 545 .

XU W D , TAN Y . Semisupervised text classification by variational autoencoder [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2020 , 31 ( 1 ): 295 - 308 .

DONOHO D L . Compressed sensing [J ] . IEEE Transactions on Information Theory , 2006 , 52 ( 4 ): 1289 - 1306 .

WANG M D , XIAO D , XIANG Y . Low-cost and confidentiality-preserving multi-image compressed acquisition and separate reconstruction for Internet of multimedia things [J ] . IEEE Internet of Things Journal , 2021 , 8 ( 3 ): 1662 - 1673 .

RAVISHANKAR S , BRESLER Y . MR image reconstruction from highly undersampled k-space data by dictionary learning [J ] . IEEE Transactions on Medical Imaging , 2011 , 30 ( 5 ): 1028 - 1041 .

TAO L , JIANG X , LIU X Z , et al . Multiscale supervised kernel dictionary learning for SAR target recognition [J ] . IEEE Transactions on Geoscience and Remote Sensing , 2020 , 58 ( 9 ): 6281 - 6297 .

QIN C , SCHLEMPER J , CABALLERO J , et al . Convolutional recurrent neural networks for dynamic MR image reconstruction [J ] . IEEE Transactions on Medical Imaging , 2019 , 38 ( 1 ): 280 - 290 .

张帅勇 , 刘美琴 , 姚超 , 等 . 分级特征反馈融合的深度图像超分辨率重建 [J ] . 自动化学报 , 2020 , 46 : 1 - 12 .

ZHANG S Y , ZHANG M Q , YAO C , et al . Hierarchical feature feedback network for depth super-resolution reconstruction [J ] . Acta Automatica Sinica , 2020 , 46 : 1 - 12 .

TAN C C , ESWARAN C . Reconstruction and recognition of face and digit images using autoencoders [J ] . Neural Computing and Applications , 2010 , 19 ( 7 ): 1069 - 1079 .

MEHTA J , MAJUMDAR A . RODEO:robust DE-aliasing autoencoder for real-time medical image reconstruction [J ] . Pattern Recognition , 2017 , 63 : 499 - 510 .

ZHOU W Z , DU H Q , MEI W B , et al . Efficient structurally-strengthened generative adversarial network for MRI reconstruction [J ] . Neurocomputing , 2021 , 422 : 51 - 61 .

DABIRI S , LU C T , HEASLIP K , et al . Semi-supervised deep learning approach for transportation mode identification using GPS trajectory data [J ] . IEEE Transactions on Knowledge and Data Engineering , 2020 , 32 ( 5 ): 1010 - 1023 .

YUAN X , GU Y , WANG Y , et al . A deep supervised learning framework for data-driven soft sensor modeling of industrial processes [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2020 , 31 ( 11 ): 4737 - 4746 .

LIU Z , LI J G , SHEN Z Q , et al . Learning efficient convolutional networks through network slimming [C ] // 2017 IEEE International Conference on Computer Vision (ICCV) . Piscataway:IEEE Press , 2017 : 2755 - 2763 .

XIANG Q , WANG X D , SONG Y F , et al . One-dimensional convolutional neural networks for high-resolution range profile recognition via adaptively feature recalibrating and automatically channel pruning [J ] . International Journal of Intelligent Systems , 2021 , 36 ( 1 ): 332 - 361 .

王晓丹 , 来杰 , 李睿 , 等 . 多层去噪极限学习机 [J ] . 吉林大学学报(工学版) , 2020 , 50 ( 3 ): 1031 - 1039 .

WANG X D , LAI J , LI R , et al . Multilayer denoising extreme learning machine [J ] . Journal of Jilin University (Engineering and Technology Edition) , 2020 , 50 ( 3 ): 1031 - 1039 .

LI S , KAWALE J , FU Y . Deep collaborative filtering via marginalized denoising auto-encoder [C ] // Proceedings of the 24th ACM International on Conference on Information and Knowledge Management . New York:ACM Press , 2015 : 811 - 820 .

LAMATA L , ALVAREZ-RODRIGUEZ U , MARTÍN-GUERRERO J D , et al . Quantum autoencoders via quantum adders with genetic algorithms [J ] . Quantum Science and Technology , 2018 , 4 ( 1 ): 014007 .

SUN Y N , XUE B , ZHANG M J , et al . A particle swarm optimization-based flexible convolutional autoencoder for image classification [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2019 , 30 ( 8 ): 2295 - 2309 .

GLOROT X , BENGIO Y . Understanding the difficulty of training deep feedforward neural networks [C ] // Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics . New York:ACM Press , 2010 : 249 - 256 .

HE K M , ZHANG X Y , REN S Q , et al . Delving deep into rectifiers:surpassing human-level performance on ImageNet classification [C ] // 2015 IEEE International Conference on Computer Vision (ICCV) . Piscataway:IEEE Press , 2015 : 1026 - 1034 .

Views

3752

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Review of unsupervised anomaly detection techniques for medical imaging

Method of unknown protocol classification based on autoencoder

Dynamic compression method for federated learning based on feature information in ISAC networks

Target visibility prediction method based on cost-sensitive CNN-BiLSTM network

Related Author

Zhao Yingcheng

Song Xiaogang

Shi Zhenghao

You Zhenzhen

Hei Xinhong

Chunxiang GU

Weisen WU

Ya’nan SHI

Related Institution

School of Computer Science and Engineering, Xi’an University of Technology

Shaanxi Key Laboratory for Network Computing and Security Technology

Human Machine Integration Intelligent Robot Shaanxi Provincial University Engineering Research Center

School of Cyberspace Security,Information Engineering University

Henan Key Laboratory of Network Cryptography Technology

AI问答

⁰