A note on the group lasso and a sparse group lasso

Abstract

We consider the group lasso penalty for the linear model. We note that the standard algorithm for solving the problem assumes that the model matrices in each group are orthonormal. Here we consider a more general penalty that blends the lasso (L1) with the group lasso ("two-norm"). This penalty yields solutions that are sparse at both the group and individual feature levels. We derive an efficient algorithm for the resulting convex problem based on coordinate descent. This algorithm can also be used to solve the general form of the group lasso, with non-orthonormal model matrices.

8 pages, 3 figs

Cited by in corpus (125)

• Theory-guided Data Science: A New Paradigm for Scientific Discovery from Data
• Structured Compressed Sensing: From Theory to Applications
• Group Sparse Regularization for Deep Neural Networks
• Proximal Methods for Hierarchical Sparse Coding
• Smoothing proximal gradient method for general structured sparse regression
• C-HiLasso: A Collaborative Hierarchical Sparse Modeling Framework
• Correlated variables in regression: clustering and sparse estimation
• Convex and Network Flow Optimization for Structured Sparsity
• Grouped Feature Importance and Combined Features Effect Plot
• Sparse Convex Clustering
• Anderson Accelerated Douglas-Rachford Splitting
• Efficient Sparse Group Feature Selection via Nonconvex Optimization
• Online Nonnegative Matrix Factorization with Outliers
• Nested Sparse Approximation: Structured Estimation of V2V Channels Using Geometry-Based Stochastic Channel Model
• Block-based Bayesian epistasis association mapping with application to WTCCC type 1 diabetes data
• Union Support Recovery in Multi-task Learning
• Sparse Activity and Sparse Connectivity in Supervised Learning
• Sparsity with sign-coherent groups of variables via the cooperative-Lasso
• Reliable recovery of hierarchically sparse signals for Gaussian and Kronecker product measurements
• Randomized Block Coordinate Descent for Online and Stochastic Optimization
• Exact block-wise optimization in group lasso and sparse group lasso for linear regression
• Auxiliary Image Regularization for Deep CNNs with Noisy Labels
• Adaptive sparse group LASSO in quantile regression
• An Online Approach to Cyberattack Detection and Localization in Smart Grid
• Sparse Range-constrained Learning and Its Application for Medical Image Grading
• Video Skimming: Taxonomy and Comprehensive Survey
• Iteration Complexity of Randomized Block-Coordinate Descent Methods for Minimizing a Composite Function
• Group Sparse Additive Models
• Recovery of Structured Signals with Prior Information via Maximizing Correlation
• Learning Efficient Structured Sparse Models
• On Sparse High-Dimensional Graphical Model Learning For Dependent Time Series
• Two-Layer Feature Reduction for Sparse-Group Lasso via Decomposition of Convex Sets
• Learning efficient sparse and low rank models
• Style Imitation and Chord Invention in Polyphonic Music with Exponential Families
• Selection of variables and decision boundaries for functional data via bi-level selection
• Learning Scale-Free Networks by Dynamic Node-Specific Degree Prior
• Semi-device-dependent blind quantum tomography
• Neither Global Nor Local: A Hierarchical Robust Subspace Clustering For Image Data
• Combinatorial Selection and Least Absolute Shrinkage via the CLASH Algorithm
• Ranked Sparsity: A Cogent Regularization Framework for Selecting and Estimating Feature Interactions and Polynomials
• Penalized integrative analysis under the accelerated failure time model
• Vector-Space Markov Random Fields via Exponential Families
• Interpreting Deep Neural Networks Through Variable Importance
• Handcrafted Local Features are Convolutional Neural Networks
• Stock Market Prediction from WSJ: Text Mining via Sparse Matrix Factorization
• Robust supervised classification and feature selection using a primal-dual method
• Individual Data Protected Integrative Regression Analysis of High-dimensional Heterogeneous Data
• Asynchronous Decentralized Successive Convex Approximation
• Hierarchical Isometry Properties of Hierarchical Measurements
• Structured Bayesian Compression for Deep Neural Networks Based on The Turbo-VBI Approach
• Efficient Mixed-Norm Regularization: Algorithms and Safe Screening Methods
• Bayesian Optimal Approximate Message Passing to Recover Structured Sparse Signals
• Hierarchical array priors for ANOVA decompositions of cross-classified data
• Learning High-Dimensional Differential Graphs From Multi-Attribute Data
• The Best of Both Worlds: Combining Data-independent and Data-driven Approaches for Action Recognition
• Bayesian Sparse Global-Local Shrinkage Regression for Selection of Grouped Variables
• Functional Linear Regression with Mixed Predictors
• Distributed Multitask Learning
• Hierarchical Sparse and Collaborative Low-Rank Representation for Emotion Recognition
• Quantum Algorithms for the Pathwise Lasso
• A Linearly Convergent Majorized ADMM with Indefinite Proximal Terms for Convex Composite Programming and Its Applications
• How Large a Vocabulary Does Text Classification Need? A Variational Approach to Vocabulary Selection
• AcSel: selecting variables with accuracy in correlated datasets
• Identifying Genetic Risk Factors via Sparse Group Lasso with Group Graph Structure
• Make Workers Work Harder: Decoupled Asynchronous Proximal Stochastic Gradient Descent
• A High-dimensional M-estimator Framework for Bi-level Variable Selection
• selectBoost: a general algorithm to enhance the performance of variable selection methods in correlated datasets
• Clustering Noisy Signals with Structured Sparsity Using Time-Frequency Representation
• Clustering of variables for enhanced interpretability of predictive models
• Variable selection via Group LASSO Approach : Application to the Cox Regression and frailty model
• Grouped Variable Selection via Nested Spike and Slab Priors
• Sparse Group Selection Through Co-Adaptive Penalties
• Factorization Machines with Regularization for Sparse Feature Interactions
• Structured Input-Output Lasso, with Application to eQTL Mapping, and a Thresholding Algorithm for Fast Estimation
• Hierarchical sparse recovery from hierarchically structured measurements with application to massive random access
• Hierarchical Maximum-Margin Clustering
• High-Dimensional Forecasting in the Presence of Unit Roots and Cointegration
• A Penalized Multi-trait Mixed Model for Association Mapping in Pedigree-based GWAS
• Efficient sparse semismooth Newton methods for the clustered lasso problem
• On Regularized Square-root Regression Problems: Distributionally Robust Interpretation and Fast Computations
• Motor-imagery classification model for brain-computer interface: a sparse group filter bank representation model
• Scalable and Accurate Online Feature Selection for Big Data
• Asgl: A Python Package for Penalized Linear and Quantile Regression
• Sparse Regression for Extreme Values
• A Unifying Framework for Typical Multi-Task Multiple Kernel Learning Problems
• Sparse-Group Bayesian Feature Selection Using Expectation Propagation for Signal Recovery and Network Reconstruction
• Feature Dimensionality Reduction for Video Affect Classification: A Comparative Study
• ABM: an automatic supervised feature engineering method for loss based models based on group and fused lasso
• r-Instance Learning for Missing People Tweets Identification
• Understanding and Monitoring Human Trafficking via Social Sensors: A Sociological Approach
• ISLET: Fast and Optimal Low-rank Tensor Regression via Importance Sketching
• Sparse group lasso and high dimensional multinomial classification
• Multiple Regression for Matrix and Vector Predictors: Models, Theory, Algorithms, and Beyond
• svReg: Structural Varying-coefficient regression to differentiate how regional brain atrophy affects motor impairment for Huntington disease severity groups
• Generalization error minimization: a new approach to model evaluation and selection with an application to penalized regression
• Efficient proximal gradient algorithms for joint graphical lasso
• Feature Clustering for Support Identification in Extreme Regions
• Instrument variable detection with graph learning : an application to high dimensional GIS-census data for house pricing
• The Complete Lasso Tradeoff Diagram
• A dual semismooth Newton based augmented Lagrangian method for large-scale linearly constrained sparse group square-root Lasso problems
• Multi-Attribute Graph Estimation with Sparse-Group Non-Convex Penalties
• Convex Latent Effect Logit Model via Sparse and Low-rank Decomposition
• Vector-valued Reproducing Kernel Banach Spaces with Group Lasso Norms
• Guaranteed blind deconvolution and demixing via hierarchically sparse reconstruction
• DebiNet: Debiasing Linear Models with Nonlinear Overparameterized Neural Networks
• Hierarchical Group Sparse Regularization for Deep Convolutional Neural Networks
• Broad Learning for Healthcare
• L-CO-Net: Learned Condensation-Optimization Network for Clinical Parameter Estimation from Cardiac Cine MRI
• The 1995-2018 Global Evolution of the Network of Amicable and Hostile Relations Among Nation-States
• A Partial Regularization Method for Network Compression
• Network Inference by Learned Node-Specific Degree Prior
• An Improving Framework of regularization for Network Compression
• A general framework for estimation and inference from clusters of features
• Estimation of Graphical Models using the $L_{1,2}$ Norm
• A novel prestack sparse azimuthal AVO inversion
• An Iterative Algorithm for Fitting Nonconvex Penalized Generalized Linear Models with Grouped Predictors
• Compressed Sensing for Block-Sparse Smooth Signals
• Hierarchical restricted isometry property for Kronecker product measurements
• Local-Aggregate Modeling for Big-Data via Distributed Optimization: Applications to Neuroimaging
• Feature selection for high-dimensional integrated data
• Reconstructing DNA copy number by joint segmentation of multiple sequences
• Estimation of a sparse group of sparse vectors
• Fast projections onto mixed-norm balls with applications
• Asymptotic Statistical Analysis of Sparse Group LASSO via Approximate Message Passing Algorithm
• Pathways-driven Sparse Regression Identifies Pathways and Genes Associated with High-density Lipoprotein Cholesterol in Two Asian Cohorts