Multi-label feature selection algorithm based on joint mutual information of max-relevance and min-redundancy

Li ZHANG; Cong WANG

doi:10.11959/j.issn.1000-436x.2018082

您当前的位置：

首页 >

文章列表页 >

Multi-label feature selection algorithm based on joint mutual information of max-relevance and min-redundancy

Papers | 更新时间：2024-06-05

- Multi-label feature selection algorithm based on joint mutual information of max-relevance and min-redundancy
- Journal on Communications Vol. 39, Issue 5, Pages: 111-122(2018)
- 作者机构：
  
  1. 北京邮电大学软件学院，北京 100876
  2. 北京邮电大学可信分布式计算与服务教育部重点实验室，北京 100876
- 作者简介：
- 基金信息：
  
  The National Science and Technology Basic Work Project(2015FY111700-6)
- DOI：10.11959/j.issn.1000-436x.2018082
  CLC： TP181
- Online First：2018-05，
  
  Published：25 May 2018
- 稿件说明：
移动端阅览
Li ZHANG, Cong WANG. Multi-label feature selection algorithm based on joint mutual information of max-relevance and min-redundancy[J]. Journal on Communications, 2018, 39(5): 111-122.
DOI：

Li ZHANG, Cong WANG. Multi-label feature selection algorithm based on joint mutual information of max-relevance and min-redundancy[J]. Journal on Communications, 2018, 39(5): 111-122. DOI： 10.11959/j.issn.1000-436x.2018082.

摘要

在过去的几十年中，特征选择已经在机器学习和人工智能领域发挥着重要作用。许多特征选择算法都存在着选择一些冗余和不相关特征的现象，这是因为它们过分夸大某些特征重要性。同时，过多的特征会减慢机器学习的速度，并导致分类过渡拟合。因此，提出新的基于前向搜索的非线性特征选择算法，该算法使用互信息和交互信息的理论，寻找与多分类标签相关的最优子集，并降低计算复杂度。在UCI中9个数据集和4个不同的分类器对比实验中表明，该算法均优于原始特征集和其他特征选择算法选择出的特征集。

Abstract

Feature selection has played an important role in machine learning and artificial intelligence in the past decades.Many existing feature selection algorithm have chosen some redundant and irrelevant features

which is leading to overestimation of some features.Moreover

more features will significantly slow down the speed of machine learning and lead to classification over-fitting.Therefore

a new nonlinear feature selection algorithm based on forward search was proposed.The algorithm used the theory of mutual information and mutual information to find the optimal subset associated with multi-task labels and reduced the computational complexity.Compared with the experimental results of nine datasets and four different classifiers in UCI

the proposed algorithm is superior to the feature set selected by the original feature set and other feature selection algorithms.

关键词

Keywords

references

GEORGE G , HAAS M R , PENTLAND A . Big data and management [J ] . Academy of Management Journal , 2014 , 57 ( 2 ): 321 - 326 .

XIE J Y , XIE W X . Several selection algorithms based on the discernibility of a feature subset and support vector machines [J ] . Chinese Journal of Computers , 2014 , 37 ( 8 ): 1704 - 1718 .

BROWN G , POCOCK A , ZHAO M J , et al . Conditional likelihood maximisation-a unifying framework for information theoretic feature selection [J ] . Journal of Machine Learning Research , 2012 , 13 : 27 - 66 .

CHENG H G , QIN Z , FENG C , et al . Conditional mutual information based feature selection analysing for synergy and redundancy [J ] . Electronics and Telecommunications Research Institute , 2011 ( 33 ): 210 - 218 .

CHANDRASHEKAR G , SAHIN F . A survey on feature selection methods [J ] . Computers and Electrical Engineering , 2014 ( 40 ): 16 - 28 .

ZHANG Z H , LI S N , LI Z G , et al . Multi-label feature selection algorithm based on information entropy [J ] . Journal of Computer Research and Development , 2013 , 50 ( 6 ): 1177 - 1184 .

YU H , YANG J . A direct LDA algorithm for high-dimensional data with application to face recognition [J ] . Pattern Recognition , 2001 ( 34 ): 2067 - 2070 .

BAJWA I S , NAWEED M S , ASIF M N , et al . Feature based image classification by using principal component analysis [J ] . ICGST International Journal on Graphics Vision and Image Processing , 2009 ( 9 ): 11 - 17 .

MALDONADO S , WEBER R . A wrapper method for feature selection using support vector machine [J ] . Information Science , 2009 , 179 ( 13 ): 2208 - 2217 .

PENG C . Distributed K-Means clustering algorithm based on Fisher discriminant ratio [J ] . Journal of Jiangsu University , 2014 , 35 ( 4 ): 422 - 427 .

ZHANG Y S , YANG A , XIONG C , et al . Feature selection using data envelopment analysis [J ] . Knowledge-Based Systems , 2014 ( 64 ): 70 - 80 .

YU L , LIU H . Feature selection for high-dimensional data:a fast correlation-based filter solution [C ] // The 20th International Conferences on machine learning . 2003 : 856 - 863 .

HUANG D , CHOW T W S . Effective feature selection scheme using mutual information [J ] . Neurocomputing , 2005 ( 63 ): 325 - 343 .

LIU H W , SUN J G , LIU L , et al . Feature selection with dynamic mutual information [J ] . IEEE Transactions on Neural Networks , 2009 , 20 ( 2 ): 189 - 201 .

DUAN H X , ZHANG Q Y , ZHANG M . FCBF algorithm based on normalized mutual information for feature selection [J ] . Journal Huazhong University of Science ＆ Technology(Natural Science Edition) , 2017 , 45 ( 1 ): 52 - 56 .

SUN G L , SONG Z C , LIU J L , et al . Feature selection method based on maximum information coefficient and approximate markov blanket [J ] . Acta Automatica Sinica , 2017 , 43 ( 5 ): 795 - 805 .

VERGARA J R , ESTEVEZ P . A review of feature selection methods based on mutual information [J ] . Neural Computing and Applications , 2014 , 24 ( 1 ): 175 - 186 .

KWAK N , CHOI C H . Input feature selection for classification problems [J ] . IEEE Transactions on Neural Networks , 2002 ( 13 ): 143 - 159 .

ESTÉVEZ P A , TESMER M , PEREZ C A , et al . Normalized mutual information feature selection [J ] . IEEE Transaction on Neural Networks , 2009 ( 20 ): 189 - 201 .

HOQUE N , BHATTACHARYYA D K , KALITA J K . MIFS-ND:a mutual information-based feature selection method [J ] . Expert Systems with Applications , 2014 , 41 ( 14 ): 6371 - 6385 .

HOWARD H Y , JOHN M . Feature selection based on joint mutual information [C ] // Advances in Intelligent Data Analysis (AIDA),Computational Intelligence Methods and Applications (CIMA),International Computer Science Conventions Rochester New York . 1999 : 1 - 8 .

PENG H , LONG F , DING C . Feature selection based on mutual information:criteria of max-dependency,max-relevance,and min- redundancy [C ] // IEEE Transaction on Pattern Analysis ＆ Machine Intelligence . 2005 , 27 ( 8 ): 1226 - 1238 .

VINH L T , THANG N D , LEE Y K . An improved maximum relevance and minimum redundancy feature selection algorithm based on normalized mutual information [C ] // Tenth International Symposium on Applications and the Internet . 2010 : 395 - 398 .

LEE J , KIM D W . Mutual information-based multi-label feature selection using interaction information [J ] . Expert Systems with Applications , 2015 ( 42 ): 2013 - 2025 .

COVER T , THOMAS J . Elements of theory [M ] . New York : John Wiley ＆ Sons , 2002 .

JAKULIN A . Attribute interactions in machine learning (Master thesis) [M ] // Lecture Notes in Computer Science . 2003 .

JOHN G H , KOHAVI R , PFLEGER K . Irrelevant features and the subset selection problem [C ] // The Eleventh International Conference on Machine Learning , 1994 : 121 - 129 .

BENNASAR M , HICKS Y , SETCHI R . Feature selection using Joint mutual information maximisation [J ] . Expert System Application , 2015 ( 42 ): 8520 - 8532 .

ZHANG Y S , ZHANG Z G . Feature subset selection with cumulate conditional mutual information minimization [J ] . Expert Systems with Applications , 2012 , 39 ( 5 ): 6078 - 6088 .

YU L , LIU H . Efficient feature selection via analysis of relevance and redundancy [J ] . Journal of Machine Learning Research , 2004 , 5 ( 12 ): 1205 - 1224 .

TAPIA E , BULACIO P , ANGELONE L F . Sparse and stable gene selection with consensus SVM-RFE [J ] . Pattern Recognition Letters , 2012 , 33 ( 2 ): 164 - 172 .

UNLER A , MURAT A , CHINNAM R B . mr2PSO:a maximum relevance minimum redundancy feature selection method based on swarm intelligence for support vector machine classification [J ] . Information Sciences , 2011 ( 20 ): 4625 - 4641 .

CHE J X , YANG Y L , LI L , et al . Maximum relevance minimum common redundancy feature selection for nonlinear data [J ] . Information Sciences , 2017 ( 5 ): 68 - 89 .

CHAKRABORTY R , PAL N R . Feature selection using a neural framework with controlled redundancy [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2015 , 26 ( 1 ): 35 - 50 .

AKADI A E , OUARDIGHI A , ABOURAJDINE D . A powerful feature selection approach based on mutual information [J ] . International Journal of Computer Science and Network Security , 2008 ( 8 ): 116 - 211 .

FLEURET F . Fast binary feature selection with conditional mutual information [J ] . Journal of Machine Learning Research , 2004 ( 5 ): 1531 - 1555 .

NIU X T . Support vector extracted algorithm based on KNN and 10 fold cross-validation method [J ] . Journal of Huazhong Normal University , 2014 , 48 ( 3 ): 335 - 338 .

Views

2864

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Research on scenario recognition for THz channels based on mRMR-GA

DGA malicious domain name identification based on XGBoost and particle swarm optimization algorithm

Abnormal traffic detection method based on multi-scale attention feature enhancement

Network intrusion detection method based on VAE-CWGAN and fusion of statistical importance of feature

Parallel deep forest algorithm based on Spark and three-way interactive information

Related Author

ZHANG Jie

LUO Jiao

LIN Feng

WANG Yang

LIAO Xi

HAO Xinyu

CHEN Weijie

FENG Lichun

Related Institution

School of Communication and Information Engineering, Chongqing University of Posts and Telecommunications

Information Technology Center, Guangzhou Academy of Fine Arts

School of Information Engineering, Yangzhou University

School of Computer Science and Technology, Civil Aviation University of China

School of Safety Science and Engineering, Civil Aviation University of China

AI问答

⁰