High-Dimensional Feature Selection by Feature-Wise Kernelized Lasso

Abstract

The goal of supervised feature selection is to find a subset of input features that are responsible for predicting output values. The least absolute shrinkage and selection operator (Lasso) allows computationally efficient feature selection based on linear dependency between input features and output values. In this paper, we consider a feature-wise kernelized Lasso for capturing non-linear input-output dependency. We first show that, with particular choices of kernel functions, non-redundant features with strong statistical dependence on output values can be found in terms of kernel-based independence measures. We then show that the globally optimal solution can be efficiently computed; this makes the approach scalable to high-dimensional problems. The effectiveness of the proposed method is demonstrated through feature selection experiments with thousands of features.

18 pages

References in corpus (2)

• Variable selection in nonparametric additive models
• Fast learning rate of multiple kernel learning: Trade-off between sparsity and smoothness

Cited by in corpus (7)

• Graph Convolutional Network-based Feature Selection for High-dimensional and Low-sample Size Data
• High-Dimensional Feature Selection for Genomic Datasets
• A User-Guided Bayesian Framework for Ensemble Feature Selection in Life Science Applications (UBayFS)
• High-Dimensional Yield Estimation using Shrinkage Deep Features and Maximization of Integral Entropy Reduction
• Recovery of Linear Components: Reduced Complexity Autoencoder Designs
• Exploring Content and Social Connections of Fake News with Explainable Text and Graph Learning
• MBExplainer: Multilevel bandit-based explanations for downstream models with augmented graph embeddings