Kernel Feature Selection via Conditional Covariance Minimization

Abstract

We propose a method for feature selection that employs kernel-based measures of independence to find a subset of covariates that is maximally predictive of the response. Building on past work in kernel dimension reduction, we show how to perform feature selection via a constrained optimization problem involving the trace of the conditional covariance operator. We prove various consistency results for this procedure, and also demonstrate that our method compares favorably with other state-of-the-art algorithms on a variety of synthetic and real data sets.

The first two authors contributed equally

References in corpus (1)

• Kernel dimension reduction in regression

Cited by in corpus (8)

• Learning to Explain: An Information-Theoretic Perspective on Model Interpretation
• Feature Importance Ranking for Deep Learning
• High-Dimensional Feature Selection for Genomic Datasets
• Differentiable Unsupervised Feature Selection based on a Gated Laplacian
• Structured nonlinear variable selection
• BPMR: Bayesian Probabilistic Multivariate Ranking
• Uncertainty-aware Cardinality Estimation by Neural Network Gaussian Process
• A Distributionally Robust Optimization Method for Adversarial Multiple Kernel Learning