基于最大相关最小冗余联合互信息的多标签特征选择算法

张俐; 王枞

doi:10.11959/j.issn.1000-436x.2018082

您当前的位置：

首页 >

文章列表页 >

基于最大相关最小冗余联合互信息的多标签特征选择算法

学术论文 | 更新时间：2024-06-05

- 基于最大相关最小冗余联合互信息的多标签特征选择算法
- Multi-label feature selection algorithm based on joint mutual information of max-relevance and min-redundancy
- 通信学报 2018年39卷第5期页码：111-122
- 作者机构：
  
  1. 北京邮电大学软件学院，北京 100876
  2. 北京邮电大学可信分布式计算与服务教育部重点实验室，北京 100876
- 作者简介：
  
  [ "张俐（1977-），男，陕西汉中人，北京邮电大学博士生，主要研究方向为机器学习、特征工程、医疗健康数据分析挖掘。" ]
  [ "王枞（1958-），女，北京人，博士，北京邮电大学教授、博士生导师，主要研究方向为智能信息处理、网络信息安全、可信计算与服务、医疗健康数据分析挖掘。" ]
- 基金信息：
  
  国家科技基础性工作专项基金资助项目(2015FY111700-6)
- DOI：10.11959/j.issn.1000-436x.2018082
  中图分类号： TP181
- 网络出版日期：2018-05，
  
  纸质出版日期：2018-05-25
- 稿件说明：
移动端阅览
张俐, 王枞. 基于最大相关最小冗余联合互信息的多标签特征选择算法[J]. 通信学报, 2018,39(5):111-122.

Li ZHANG, Cong WANG. Multi-label feature selection algorithm based on joint mutual information of max-relevance and min-redundancy[J]. Journal on communications, 2018, 39(5): 111-122.
张俐, 王枞. 基于最大相关最小冗余联合互信息的多标签特征选择算法[J]. 通信学报, 2018,39(5):111-122. DOI： 10.11959/j.issn.1000-436x.2018082.

Li ZHANG, Cong WANG. Multi-label feature selection algorithm based on joint mutual information of max-relevance and min-redundancy[J]. Journal on communications, 2018, 39(5): 111-122. DOI： 10.11959/j.issn.1000-436x.2018082.

摘要

在过去的几十年中，特征选择已经在机器学习和人工智能领域发挥着重要作用。许多特征选择算法都存在着选择一些冗余和不相关特征的现象，这是因为它们过分夸大某些特征重要性。同时，过多的特征会减慢机器学习的速度，并导致分类过渡拟合。因此，提出新的基于前向搜索的非线性特征选择算法，该算法使用互信息和交互信息的理论，寻找与多分类标签相关的最优子集，并降低计算复杂度。在UCI中9个数据集和4个不同的分类器对比实验中表明，该算法均优于原始特征集和其他特征选择算法选择出的特征集。

Abstract

Feature selection has played an important role in machine learning and artificial intelligence in the past decades.Many existing feature selection algorithm have chosen some redundant and irrelevant features

which is leading to overestimation of some features.Moreover

more features will significantly slow down the speed of machine learning and lead to classification over-fitting.Therefore

a new nonlinear feature selection algorithm based on forward search was proposed.The algorithm used the theory of mutual information and mutual information to find the optimal subset associated with multi-task labels and reduced the computational complexity.Compared with the experimental results of nine datasets and four different classifiers in UCI

the proposed algorithm is superior to the feature set selected by the original feature set and other feature selection algorithms.

关键词

Keywords

references

GEORGE G , HAAS M R , PENTLAND A . Big data and management [J ] . Academy of Management Journal , 2014 , 57 ( 2 ): 321 - 326 .

XIE J Y , XIE W X . Several selection algorithms based on the discernibility of a feature subset and support vector machines [J ] . Chinese Journal of Computers , 2014 , 37 ( 8 ): 1704 - 1718 .

BROWN G , POCOCK A , ZHAO M J , et al . Conditional likelihood maximisation-a unifying framework for information theoretic feature selection [J ] . Journal of Machine Learning Research , 2012 , 13 : 27 - 66 .

CHENG H G , QIN Z , FENG C , et al . Conditional mutual information based feature selection analysing for synergy and redundancy [J ] . Electronics and Telecommunications Research Institute , 2011 ( 33 ): 210 - 218 .

CHANDRASHEKAR G , SAHIN F . A survey on feature selection methods [J ] . Computers and Electrical Engineering , 2014 ( 40 ): 16 - 28 .

ZHANG Z H , LI S N , LI Z G , et al . Multi-label feature selection algorithm based on information entropy [J ] . Journal of Computer Research and Development , 2013 , 50 ( 6 ): 1177 - 1184 .

YU H , YANG J . A direct LDA algorithm for high-dimensional data with application to face recognition [J ] . Pattern Recognition , 2001 ( 34 ): 2067 - 2070 .

BAJWA I S , NAWEED M S , ASIF M N , et al . Feature based image classification by using principal component analysis [J ] . ICGST International Journal on Graphics Vision and Image Processing , 2009 ( 9 ): 11 - 17 .

MALDONADO S , WEBER R . A wrapper method for feature selection using support vector machine [J ] . Information Science , 2009 , 179 ( 13 ): 2208 - 2217 .

PENG C . Distributed K-Means clustering algorithm based on Fisher discriminant ratio [J ] . Journal of Jiangsu University , 2014 , 35 ( 4 ): 422 - 427 .

ZHANG Y S , YANG A , XIONG C , et al . Feature selection using data envelopment analysis [J ] . Knowledge-Based Systems , 2014 ( 64 ): 70 - 80 .

YU L , LIU H . Feature selection for high-dimensional data:a fast correlation-based filter solution [C ] // The 20th International Conferences on machine learning . 2003 : 856 - 863 .

HUANG D , CHOW T W S . Effective feature selection scheme using mutual information [J ] . Neurocomputing , 2005 ( 63 ): 325 - 343 .

LIU H W , SUN J G , LIU L , et al . Feature selection with dynamic mutual information [J ] . IEEE Transactions on Neural Networks , 2009 , 20 ( 2 ): 189 - 201 .

DUAN H X , ZHANG Q Y , ZHANG M . FCBF algorithm based on normalized mutual information for feature selection [J ] . Journal Huazhong University of Science ＆ Technology(Natural Science Edition) , 2017 , 45 ( 1 ): 52 - 56 .

SUN G L , SONG Z C , LIU J L , et al . Feature selection method based on maximum information coefficient and approximate markov blanket [J ] . Acta Automatica Sinica , 2017 , 43 ( 5 ): 795 - 805 .

VERGARA J R , ESTEVEZ P . A review of feature selection methods based on mutual information [J ] . Neural Computing and Applications , 2014 , 24 ( 1 ): 175 - 186 .

KWAK N , CHOI C H . Input feature selection for classification problems [J ] . IEEE Transactions on Neural Networks , 2002 ( 13 ): 143 - 159 .

ESTÉVEZ P A , TESMER M , PEREZ C A , et al . Normalized mutual information feature selection [J ] . IEEE Transaction on Neural Networks , 2009 ( 20 ): 189 - 201 .

HOQUE N , BHATTACHARYYA D K , KALITA J K . MIFS-ND:a mutual information-based feature selection method [J ] . Expert Systems with Applications , 2014 , 41 ( 14 ): 6371 - 6385 .

HOWARD H Y , JOHN M . Feature selection based on joint mutual information [C ] // Advances in Intelligent Data Analysis (AIDA),Computational Intelligence Methods and Applications (CIMA),International Computer Science Conventions Rochester New York . 1999 : 1 - 8 .

PENG H , LONG F , DING C . Feature selection based on mutual information:criteria of max-dependency,max-relevance,and min- redundancy [C ] // IEEE Transaction on Pattern Analysis ＆ Machine Intelligence . 2005 , 27 ( 8 ): 1226 - 1238 .

VINH L T , THANG N D , LEE Y K . An improved maximum relevance and minimum redundancy feature selection algorithm based on normalized mutual information [C ] // Tenth International Symposium on Applications and the Internet . 2010 : 395 - 398 .

LEE J , KIM D W . Mutual information-based multi-label feature selection using interaction information [J ] . Expert Systems with Applications , 2015 ( 42 ): 2013 - 2025 .

COVER T , THOMAS J . Elements of theory [M ] . New York : John Wiley ＆ Sons , 2002 .

JAKULIN A . Attribute interactions in machine learning (Master thesis) [M ] // Lecture Notes in Computer Science . 2003 .

JOHN G H , KOHAVI R , PFLEGER K . Irrelevant features and the subset selection problem [C ] // The Eleventh International Conference on Machine Learning , 1994 : 121 - 129 .

BENNASAR M , HICKS Y , SETCHI R . Feature selection using Joint mutual information maximisation [J ] . Expert System Application , 2015 ( 42 ): 8520 - 8532 .

ZHANG Y S , ZHANG Z G . Feature subset selection with cumulate conditional mutual information minimization [J ] . Expert Systems with Applications , 2012 , 39 ( 5 ): 6078 - 6088 .

YU L , LIU H . Efficient feature selection via analysis of relevance and redundancy [J ] . Journal of Machine Learning Research , 2004 , 5 ( 12 ): 1205 - 1224 .

TAPIA E , BULACIO P , ANGELONE L F . Sparse and stable gene selection with consensus SVM-RFE [J ] . Pattern Recognition Letters , 2012 , 33 ( 2 ): 164 - 172 .

UNLER A , MURAT A , CHINNAM R B . mr2PSO:a maximum relevance minimum redundancy feature selection method based on swarm intelligence for support vector machine classification [J ] . Information Sciences , 2011 ( 20 ): 4625 - 4641 .

CHE J X , YANG Y L , LI L , et al . Maximum relevance minimum common redundancy feature selection for nonlinear data [J ] . Information Sciences , 2017 ( 5 ): 68 - 89 .

CHAKRABORTY R , PAL N R . Feature selection using a neural framework with controlled redundancy [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2015 , 26 ( 1 ): 35 - 50 .

AKADI A E , OUARDIGHI A , ABOURAJDINE D . A powerful feature selection approach based on mutual information [J ] . International Journal of Computer Science and Network Security , 2008 ( 8 ): 116 - 211 .

FLEURET F . Fast binary feature selection with conditional mutual information [J ] . Journal of Machine Learning Research , 2004 ( 5 ): 1531 - 1555 .

NIU X T . Support vector extracted algorithm based on KNN and 10 fold cross-validation method [J ] . Journal of Huazhong Normal University , 2014 , 48 ( 3 ): 335 - 338 .

浏览量

2037

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于XGBoost和粒子群优化算法的DGA恶意域名识别

基于多尺度注意力特征增强的异常流量检测方法

基于VAE-CWGAN和特征统计重要性融合的网络入侵检测方法

基于Spark和三路交互信息的并行深度森林算法

联合低秩重构和投影重构的稳健特征选择方法