A Review of Feature Selection Methods Based on Mutual Information

Abstract

In this work we present a review of the state of the art of information theoretic feature selection methods. The concepts of feature relevance, redundance and complementarity (synergy) are clearly defined, as well as Markov blanket. The problem of optimal feature selection is defined. A unifying theoretical framework is described, which can retrofit successful heuristic criteria, indicating the approximations made by each method. A number of open problems in the field are presented.

References in corpus (1)

• Quantifying and Visualizing Attribute Interactions

Cited by in corpus (50)

• Big-Data Science in Porous Materials: Materials Genomics and Machine Learning
• Towards Stacking Fault Energy Engineering in FCC High Entropy Alloys
• Computational Intelligence Challenges and Applications on Large-Scale Astronomical Time Series Databases
• Feature Selection and Feature Extraction in Pattern Analysis: A Literature Review
• An investigation on the factors affecting machine learning classifications in $\gamma$-ray astronomy
• Employee turnover prediction and retention policies design: a case study
• Relevant based structure learning for feature selection
• Limits on Inferring T-cell Specificity from Partial Information
• Feature Selection: A perspective on inter-attribute cooperation
• Data Compression and Inference in Cosmology with Self-Supervised Machine Learning
• Data augmentation and feature selection for automatic model recommendation in computational physics
• Multivariate Feature Selection and Autoencoder Embeddings of Ovarian Cancer Clinical and Genetic Data
• Feature Selection from High-Dimensional Data with Very Low Sample Size: A Cautionary Tale
• Understanding Convolutional Neural Networks with Information Theory: An Initial Exploration
• A representation learning approach to probe for dynamical dark energy in matter power spectra
• Kernel Partial Correlation Coefficient -- a Measure of Conditional Dependence
• Simple stopping criteria for information theoretic feature selection
• Ensemble Estimation of Generalized Mutual Information with Applications to Genomics
• Feature selection for medical diagnosis: Evaluation for using a hybrid Stacked-Genetic approach in the diagnosis of heart disease
• PIKS: A Technique to Identify Actionable Trends for Policy-Makers Through Open Healthcare Data
• Learned practical guidelines for evaluating Conditional Entropy and Mutual Information in discovering major factors of response-vs-covariate dynamics
• A Rigorous Information-Theoretic Definition of Redundancy and Relevancy in Feature Selection Based on (Partial) Information Decomposition
• Deep learning insights into non-universality in the halo mass function
• Geometric Estimation of Multivariate Dependency
• Enhancing ensemble learning and transfer learning in multimodal data analysis by adaptive dimensionality reduction
• AGN selection in the AKARI NEP deep field with the fuzzy SVM algorithm
• Outlier Detection as Instance Selection Method for Feature Selection in Time Series Classification
• $\Lambda$CDM and early dark energy in latent space: a data-driven parametrization of the CMB temperature power spectrum
• One-class classifiers based on entropic spanning graphs
• SparseEMG: Computational Design of Sparse EMG Layouts for Sensing Gestures
• Benchmarking Relief-Based Feature Selection Methods for Bioinformatics Data Mining
• On the Self-Penalization Phenomenon in Feature Selection
• Efficiently Estimating Mutual Information Between Attributes Across Tables
• Bounds on mutual information of mixture data for classification tasks
• Feature selection in machine learning: Rényi min-entropy vs Shannon entropy
• kD-STR: A Method for Spatio-Temporal Data Reduction and Modelling
• Mutual Information-Based Unsupervised Feature Transformation for Heterogeneous Feature Subset Selection
• Theoretical Foundations of Forward Feature Selection Methods based on Mutual Information
• Feature Selection for multi-labeled variables via Dependency Maximization
• Quantum Adiabatic Feature Selection
• Feature Dimensionality Reduction for Video Affect Classification: A Comparative Study
• Theoretical Evaluation of Feature Selection Methods based on Mutual Information
• A unifying Bayesian approach for preterm brain-age prediction that models EEG sleep transitions over age
• Approximately Optimal Binning for the Piecewise Constant Approximation of the Normalized Unexplained Variance (nUV) Dissimilarity Measure
• An Enhanced Ad Event-Prediction Method Based on Feature Engineering
• Distributed Differentially Private Mutual Information Ranking and Its Applications
• Learning Whole-Body Human-Robot Haptic Interaction in Social Contexts
• An Approximation Scheme for Multivariate Information based on Partial Information Decomposition
• TCMI: a non-parametric mutual-dependence estimator for multivariate continuous distributions
• Active feature selection discovers minimal gene sets for classifying cell types and disease states with single-cell mRNA-seq data