智能启发算法在机器学习中的应用研究综述

沈焱萍; 郑康锋; 伍淳华; 杨义先

doi:10.11959/j.issn.1000-436x.2019242

您当前的位置：

首页 >

文章列表页 >

智能启发算法在机器学习中的应用研究综述

综述 | 更新时间：2024-06-05

- 智能启发算法在机器学习中的应用研究综述
- Survey of research on application of heuristic algorithm in machine learning
- 通信学报 2019年40卷第12期页码：124-137
- 作者机构：
  
  1. 北京邮电大学网络空间安全学院，北京 100876
  2. 防灾科技学院信息工程学院，河北廊坊 065201
- 作者简介：
  
  [ "沈焱萍（1986– ），女，河北廊坊人，北京邮电大学博士生，防灾科技学院讲师，主要研究方向为网络与信息安全" ]
  [ "郑康锋（1975– ），男，山东烟台人，博士，北京邮电大学副教授、博士生导师，主要研究方向为网络与信息安全、量子通信等" ]
  [ "伍淳华（1976– ），女，湖北黄冈人，博士，北京邮电大学讲师，主要研究方向为人工智能、网络与信息安全" ]
  [ "杨义先（1961– ），男，四川盐亭人，博士，北京邮电大学教授、博士生导师，主要研究方向为信息安全、密码学" ]
- 基金信息：
  
  国家自然科学基金资助项目(61602052);国家重点研发计划基金资助项目(2017YFB0802803)
- DOI：10.11959/j.issn.1000-436x.2019242
  中图分类号： TP39
- 网络首发：2019-12，
  
  纸质出版：2019-12-25
- 稿件说明：
移动端阅览
沈焱萍, 郑康锋, 伍淳华, 等. 智能启发算法在机器学习中的应用研究综述[J]. 通信学报, 2019,40(12):124-137.

Yanping SHEN, Kangfeng ZHENG, Chunhua WU, et al. Survey of research on application of heuristic algorithm in machine learning[J]. Journal on Communications, 2019, 40(12): 124-137.
沈焱萍, 郑康锋, 伍淳华, 等. 智能启发算法在机器学习中的应用研究综述[J]. 通信学报, 2019,40(12):124-137. DOI： 10.11959/j.issn.1000-436x.2019242.

Yanping SHEN, Kangfeng ZHENG, Chunhua WU, et al. Survey of research on application of heuristic algorithm in machine learning[J]. Journal on Communications, 2019, 40(12): 124-137. DOI： 10.11959/j.issn.1000-436x.2019242.

摘要

针对机器学习算法在应用中存在的问题，构建基于智能启发算法的机器学习模型优化体系。首先，介绍已有智能启发算法类型及其建模过程。然后，从智能启发算法在机器学习算法中的应用，包括神经网络等参数结构优化、特征优化、集成约简、原型优化、加权投票集成和核函数学习等方面说明智能启发算法的优势。最后，结合实际需求展望智能启发算法及在机器学习领域的发展方向。

Abstract

Aiming at the problems existing in the application of machine learning algorithm

an optimization system of the machine learning model based on the heuristic algorithm was constructed.Firstly

the existing types of heuristic algorithms and the modeling process of heuristic algorithms were introduced.Then

the advantages of the heuristic algorithm were illustrated from its applications in machine learning

including the parameter and structure optimization of neural network and other machine learning algorithms

feature optimization

ensemble pruning

prototype optimization

weighted voting ensemble and kernel function learning.Finally

the heuristic algorithms and their development directions in the field of machine learning were given according to the actual needs.

关键词

Keywords

references

MITCHELL T.M . Machine learning [M ] . New York : McGraw-Hill EducationPress , 1997 .

KIM P . Deep learning for beginners:with Matlab examples [M ] . Charleston : Create Space Independent Publishing PlatformPress , 2016 .

郑泽宇 , 顾思宇 . Tensorflow:实战Google深度学习框架 [M ] . 北京 : 电子工业出版社 , 2017 .

ZHENG Z Y , GU S Y . Tensorflow:a practical Google deep learning framework [M ] . Beijing : Publishing House of Electronics IndustryPress , 2017 .

KARAMI A , GUERRERO-ZAPATA M . A fuzzy anomaly detection system based on hybrid PSO-Kmeans algorithm in content-centric networks [J ] . Neurocomputing , 2015 , 149 : 1253 - 1269 .

YANG X S , GANDOMI A H . Bat algorithm:a novel approach for global engineering optimization [J ] . Engineering Computations , 2012 , 29 ( 5 ): 464 - 483 .

YANG X S , HE X . Firefly algorithm:recent advances and applications [J ] . International Journal of Swarm Intelligence , 2013 , 1 ( 1 ):36.

MIRJALILI S . Dragonfly algorithm:a new meta-heuristic optimization technique for solving single-objective,discrete,and multi-objective problems [J ] . Neural Computing ＆ Applications , 2016 , 27 ( 4 ): 1053 - 1073 .

MIRJALILI S , LEWIS A . The whale optimization algorithm [J ] . Advances in Engineering Software , 2016 , 95 : 51 - 67 .

MIRJALILI S , MIRJALILI S M , LEWIS A . Grey wolf optimization [J ] . Advances in engineering software , 2014 , 69 ( 7 ): 46 - 61 .

MIRJALILI S . The ant lion optimizer [J ] . Advances in Engineering Software , 2015 , 83 : 80 - 98 .

MIRJALILI S , GANDOMI A H , MIRJALILI S Z , et al . Salp swarmalgorithm:a bio-inspired optimizer for engineering design problems [J ] . Advances in Engineering Software , 2017 : 163 - 191 .

TALBI E G . Hybrid Metaheuristics [M ] . Berlin : SpringerPress , 2013 : 3 - 76 .

Davis L , . Bit-climbing,representational bias,and test suite design [C ] // ICGA . CA , 1991 : 18 - 23 .

KIRKPATRICK S , JR D G , VECCHI M P . Optimization by simulated annealing [J ] . Science , 1983 , 220 : 671 - 680 .

GLOVER F . Tabu search - Part I [J ] . ORSA Journal on Computing , 1989 , 1 ( 3 ): 190 - 206 .

GLOVER F . Tabu search - Part II [J ] . ORSA Journal on Computing , 1990 , 2 ( 1 ): 4 - 32 .

KENNEDY J , EBERHART R . Particle swarm optimization [C ] // The 1995 IEEE international conference on neural networks . IEEE , 1995 : 1942 - 1948 .

HOLLAND J H . Genetic algorithms [J ] . Scientific American , 1992 , 267 ( 1 ): 66 - 72 .

RECHENBERG I , EIGEN M . Evolutions strategie:optimierung technischer systeme nach prinzipien der biologischen evolution [M ] . Stuttgart : Frommann-HolzboogPress , 1973 .

KOZA J R . Genetic programming as a means for programming computers by natural selection [J ] . Statistics ＆ Computing , 1994 , 4 ( 2 ): 87 - 112 .

STORN R , PRICE K V . Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces [J ] . Journal of Global Optimization , 1997 , 11 ( 10 ): 341 - 359 .

RASHEDI E , NEZAMABADI-POUR H , SARYAZDI S . GSA:a gravitational search algorithm [J ] . Information Science , 2009 , 179 ( 13 ): 2232 - 2248 .

KAVEH A , TALATAHARI S . A novel heuristic optimization method:charged system search [J ] . Acta Mechanica , 2010 , 213 ( 3-4 ): 267 - 289 .

FORMATO R A . Central force optimization:a new metaheuristic with applications in applied electromagnetics [J ] . Progress in Electromagnetics Research , 2007 , 77 : 425 - 491 .

EROL O K , EKSIN I . A new optimization method:big bang-big crunch [J ] . Advances in Engineering Software , 2006 , 37 ( 2 ): 106 - 111 .

DORIGO M , BIRATTARI M , STUTZLE T . Ant colony optimization [J ] . IEEE Computational Intelligence Magazine , 2007 , 1 ( 4 ): 28 - 39 .

BASTURK B , KARABOGA D . An artificial bee colony(ABC) algorithm for numeric function optimization [C ] // The IEEE swarm intelligence symposium . IEEE , 2006 : 12 - 14 .

YANG X S , DEB S . Cuckoo search via Levy flights [C ] // 2009 World Congress on Nature ＆ Biologically Inspired Computing . IEEE , 2009 : 210 - 214 .

RAO R V , SAVSANI V J , VAKHARIA D P . Teaching-learning-based optimization:an optimization method for continuous non-linear large scale problems [J ] . Information Sciences , 2012 , 183 ( 1 ): 1 - 15 .

GEEM Z W , KIM J H , LOGANATHAN G V . A new heuristic optimization algorithm:harmony search [J ] . Simulation , 2001 , 76 ( 2 ): 60 - 68 .

HE S , WU Q H , SAUNDERS J R . Group search optimizer:an optimization algorithm inspired by animal searching behavior [J ] . IEEE Transactions on Evolutionary Computation , 2009 , 13 ( 5 ): 973 - 990 .

WOLPERT D H , MACREADY W G . No free lunch theorems for optimization [J ] . IEEE Transactions on Evolutionary Computation , 1997 , 1 ( 1 ): 67 - 82 .

KOZA J R . Genetic programming:a paradigm for genetically breeding populations of computer programs to solve problems [D ] . Stanford:Stanford University , 1990 .

MAHDAVI S , SHIRI M E , RAHNAMAYAN S . Metaheuristics in large-scale global continues optimization:a survey [J ] . Information Sciences , 2015 , 295 : 407 - 428 .

ALI M , MHOAMMED B A D , ISMAIL A , et al . A new intrusion detection system based on fast learning network and particle swarm optimization [J ] . IEEE Access , 2018 ,PP( 99 ):1

XIA H , HOI S C H . MKBoost:a framework of multiple kernel boosting [J ] . IEEE Transactions on Knowledge and Data Engineering , 2013 , 25 ( 7 ): 1574 - 1586 .

YOUNG S R , ROSE D C , KARNOWSKI T P , et al . Optimizing deep learning hyper-parameters through an evolutionary algorithm [C ] // The Workshop on Machine Learning in High-Performance Computing Environments . ACM , 2015 : 1 - 5 .

AHILA R , SADASIVAM V , MANIMALA K . An integrated PSO for parameter determination and feature selection of ELM and its application in classification of power system disturbances [J ] . Applied Soft Computing , 2015 , 32 : 23 - 37 .

ZHU Q , QIN A , SUGANTHAN P , et al . Evolutionary extreme learning machine [J ] . Pattern Recognition , 2005 , 38 ( 10 ): 1759 - 1763 .

ZHANG Y , WU J , CAI Z , et al . Memetic extreme learning machine [J ] . Pattern Recognition , 2016 , 58 : 135 - 148 .

CAMILLERI M , NERI F , PAPOUTSIDAKIS M . An algorithmic approach to parameter selection in machine learning using meta-optimization techniques [J ] . Wseas Transactions on Systems , 2014 , 13 : 203 - 212 .

KARDAN A A , KAVIAN A , ESMAEILI A . Simultaneous feature selection and feature weighting with K selection for KNN classification using BBO algorithm [C ] // Information ＆ Knowledge Technology . IEEE , 2013 : 349 - 354 .

COSTA K A , PEREIRA L A , NAKAMURA R Y , et al . A nature-inspired approach to speed up optimum-path forest clustering and its application to intrusion detection in computer networks [J ] . Information Sciences , 2015 , 294 : 95 - 108 .

GHEISARI S , MEYBODI M R . BNC-PSO:structure learning of Bayesian networks by particle swarm optimization [J ] . Information Sciences , 2016 , 348 : 272 - 289 .

HUANG C L , DUN J F . A distributed PSO-SVM hybrid system with feature selection and parameter optimization [J ] . Applied Soft Computing , 2008 , 8 ( 4 ): 1381 - 1391 .

BHADRA T , BANDYOPADHYAY S , MAULIK U . Differential evolution based optimization of SVM parameters for meta classifier design [J ] . Procedia Technology , 2012 ( 4 ): 50 - 57 .

DING S , LI H , SU C , et al . Evolutionary artificial neural networks:a review [J ] . Artificial Intelligence Review , 2013 , 39 ( 3 ): 251 - 260 .

PANDAY D , RENATO C D A , LANE P . Feature weighting as a tool for unsupervised feature selection [J ] . Information Processing Letters , 2018 , 129 : 44 - 52 .

姚旭 , 王晓丹 , 张玉玺 , 等 . 特征选择方法综述 [J ] . 控制与决策 , 2012 , 27 ( 2 ): 161 - 166 .

YAO X , WANG X D , ZHANG Y X , et al . Summary of feature selection algorithms [J ] . Control and Decision , 2012 , 27 ( 2 ): 161 - 166 .

KASHEF S , NEZAMABADI-POUR H . An advanced ACO algorithm for feature subset selection [J ] . Neurocomputing , 2015 , 147 : 271 - 279 .

ZELENKOV Y , FEDOROVA E , CHEKRIZOV D . Two-step classification method based on genetic algorithm for bankruptcy forecasting [J ] . Expert Systems with Applications , 2017 , 88 : 393 - 401 .

DIAO R , SHEN Q . Nature inspired feature selection meta-heuristics [J ] . Artificial Intelligence Review , 2015 , 44 ( 3 ): 311 - 340 .

MATEOS G , GUTIRREZ J , SANTOS J C . On the evolutionary weighting of neighbors and features in the k-nearest neighbor rule [J ] . Neurocomputing , 2019 , 326-327 : 54 - 60 .

BHARTI K K , SINGH P K . Opposition chaotic fitness mutation based adaptive inertia weight BPSO for feature selection in text clustering [J ] . Applied Soft Computing , 2016 , 43 : 20 - 34 .

LI A , CHEN H L , HUANG H , et al . An enhanced grey wolf optimization based feature selection wrapped kernel extreme learning machine for medical diagnosis [J ] . Computational and Mathematical Methods in Medicine , 2017 , 2017 : 1 - 15 .

ZHANG L , MISTRY K , LIM C P , et al . Feature selection using firefly optimization for classification and regression models [J ] . Decision Support Systems , 2018 , 106 : 64 - 85 .

CRUZ R M O , SABOURIN R , CAVALCANTI G D C , et al . META-DES:a dynamic ensemble selection framework using meta-learning [J ] . Pattern Recognition , 2015 , 48 ( 5 ): 1925 - 1935 .

ZHOU Z H , WU J X , TANG W . Ensembling neural networks:many could be better than all [J ] . Artificial Intelligence , 2002 , 137 ( 1 ): 239 - 263 .

DIAO R , CHAO F , PENG T , et al . Feature selection inspired classier ensemble reduction [J ] . IEEE Transactions on Cybernetics , 2013 , 44 : 1259 - 1268 .

SHEN Y , ZHENG K , WU C , et al . An ensemble method based on selection using bat algorithm for intrusion detection [J ] . The Computer Journal , 2017 : 1 - 13 .

KRAWCZYK B . One-class classifier ensemble pruning and weighting with firefly algorithm [J ] . Neurocomputing , 2015 , 150 : 490 - 500 .

HU W M , GAO J , WANG Y , et al . Online AdaBoost-based parameterized methods for dynamic distributed network intrusion detection [J ] . IEEE Transactions on Cybernetics , 2013 , 44 ( 1 ): 66 - 82 .

ZHANG L , SRISUKKHAM W , NEOH S C , et al . Classifier ensemble reduction using a modified firefly algorithm:an empirical evaluation [J ] . Expert Systems with Applications , 2017 :93.

GARCIA S , DERRAC J , CANO J , et al . Prototype selection for nearest neighbor classification:taxonomy and empirical study [J ] . IEEE Transactions on Pattern Analysis ＆ Machine Intelligence , 2012 , 34 ( 3 ): 417 - 435 .

TRIGUERO I , GARCIA S , HERRERA F . A taxonomy and experimental study on prototype generation for nearest neighbor classification [J ] . IEEE Transactions on Systems Man ＆ Cybernetics Part C Applications ＆ Reviews , 2012 , 42 ( 1 ): 86 - 100 .

VERBIESTA N , CORNELISA C , HERRERAB F . FRPS:a fuzzy rough prototype selection method [J ] . Pattern Recognition , 2013 , 46 ( 10 ): 2770 - 2782 .

RICO J R , INESTA J M . New rank methods for reducing the size of the training set using the nearest neighbor rule [J ] . Pattern Recognition Letters , 2012 , 33 ( 5 ): 654 - 660 .

KOHONEN T . The self-organizative map [J ] . Proceedings of the IEEE , 1990 , 78 ( 9 ): 1464 - 1480 .

HAMAMOTO Y , UCHIMURA S , TOMITA S . A bootstrap technique for nearest neighbor classifier design [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 1997 , 19 ( 1 ): 73 - 79 .

LOZANO M , SOTOCA J M , SÁNCHEZ J S , et al . Experimental study on prototype optimization algorithms for prototype-based classification in vector spaces [J ] . Pattern Recognition , 2006 , 39 ( 10 ): 1827 - 1838 .

NANNI L , LUMINI A . Particle swarm optimization for prototype reduction [J ] . Neurocomputing , 2009 , 72 ( 4 ): 1092 - 1097 .

HU W W , TAN Y . Prototype generation using multiobjective particle swarm optimization for nearest neighbor classification [J ] . IEEE Transactions on Cybernetics , 2015 , 46 ( 12 ): 2719 - 2731 .

TRIGUERO I , GARCÍA S , HERRERA F . Differential evolution for optimizing the positioning of prototypes in nearest neighbor classification [J ] . Pattern Recognition , 2011 , 44 ( 4 ): 901 - 916 .

REZAEI M , NEZAMABADI-POUR H . Using gravitational search algorithm in prototype generation for nearest neighbor classification [J ] . Neurocomputing , 2015 , 157 : 256 - 263 .

PEREZ-RODRIGUEZ J, A G,GARCIA-PEDRAJAS N ， ARROYO-PEÑA A G GARCIA-PEDRAJAS N . Simultaneous instance and feature selection and weighting using evolutionary computation:Proposal and study [J ] . Applied Soft Computing , 2015 , 37 ( C ): 416 - 443 .

DERRAC J , TRIGUERO I , GARCIA S , et al . Integrating instance selection,instance weighting,and feature weighting for nearest neighbor classifiers by coevolutionary algorithms [J ] . IEEE Transactions on Systems,Man,and Cybernetics,Part B (Cybernetics) , 2012 , 42 ( 5 ): 1383 - 1397 .

ESCALANTE H J , GRAFF M , MORALES-REYES A . PGGP:prototype generation via genetic programming [J ] . Applied Soft Computing , 2016 , 40 : 569 - 580 .

VERBIEST N , VLUYMANS S , CORNELIS C , et al . Improving nearest neighbor classification using ensembles of evolutionary generated prototype subsets [J ] . Applied Soft Computing , 2016 , 44 ( C ): 75 - 88 .

CAO J , KWONG S , WANG R , et al . Class-specific soft voting based multiple extreme learning machines ensemble [J ] . Neurocomputing , 2015 , 149 : 275 - 284 .

ABUROMMAN A A , REAZ M B I . A novel SVM-kNN-PSO ensemble method for intrusion detection system [J ] . Applied Soft Computing , 2016 , 38 : 360 - 372 .

ZHANG Y , ZHANG H R , CAI J , et al . A weighted voting classifier based on differential evolution [J ] . Abstract and Applied Analysis , 2014 , 2014 : 1 - 6 .

ONAN A , KORUKOGLU S , BULUT H . A multiobjective weighted voting ensemble classifier based on differential evolution algorithm for text sentiment classification [J ] . Expert Systems with Applications , 2016 , 62 : 1 - 16 .

EKBAL A , SAHA S . Weighted vote-based classifier ensemble for named entity recognition:a genetic algorithm-based approach [J ] . ACM Transactions on Asian Language Information Processing , 2011 , 10 ( 2 ): 1 - 37 .

MA C , OUYANG J , CHEN H L , et al . A novel kernel extreme learning machine algorithm based on self-adaptive artificial bee colony optimization strategy [J ] . International Journal of Systems Science , 2015 : 1 - 16 .

GAUTHAMA RAMAN M R , SOMU N , KIRTHIVASAN K , et al . An efficient intrusion detection system based on hypergraph-genetic algorithm for parameter optimization and feature selection in support vector machine [J ] . Knowledge-Based Systems , 2017 , 134 : 1 - 12 .

ZHANG X L , CHEN X F , HE Z J . An ACO-based algorithm for parameter optimization of support vector machines [J ] . Expert Systems with Applications , 2010 , 37 ( 9 ): 6618 - 6628 .

KUANG F , XU W , ZHANG S . A novel hybrid KPCA and SVM with GA model for intrusion detection [J ] . Applied Soft Computing , 2014 , 18 ( C ): 178 - 184 .

KUANG F , ZHANG S , JIN Z , et al . A novel SVM by combining kernel principal component analysis and improved chaotic particle swarm optimization for intrusion detection [J ] . Soft Computing , 2015 , 19 ( 5 ): 1187 - 1199 .

BAMAKAN S M H , WANG H , YINGJIE T , et al . An effective intrusion detection framework based on MCLP/SVM optimized by time-varying chaos particle swarm optimization [J ] . Neurocomputing , 2016 , 199 ( C ): 90 - 102 .

BAO Y , HU Z , XIONG T . A PSO and pattern search based memetic algorithm for SVMs parameters optimization [J ] . Neurocomputing , 2013 , 117 : 98 - 106 .

AVCI D , DOGANTEKIN A . An expert diagnosis system for Parkinson disease based on genetic algorithm-wavelet kernel-extreme learning machine [J ] . Parkinson’s Disease , 2016 : 1 - 9 .

NIAZMARDI S , SAFARI A , HOMAYOUNI S . A novel multiple kernel learning framework for multiple feature classification [J ] . IEEE Journal of Selected Topics in Applied Earth Observations ＆ Remote Sensing , 2017 , 10 ( 8 ): 3734 - 3743 .

NANDA S J , PANDA G . A survey on nature inspired metaheuristic algorithms for partitional clustering [J ] . Swarm ＆ Evolutionary Computation , 2014 , 16 : 1 - 18 .

YANG X S . Nature-inspired algorithms and applied optimization [M ] . Berlin : SpringerPress , 2018 : 1 - 25 .

FONG S , DEB S , YANG X S . How meta-heuristic algorithms contribute to deep learning in the hype of big data analytics [C ] // Advances in Intelligent Systems and Computing . Springer , 2018 : 3 - 25 .

浏览量

3173

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

暂无数据