C-HiLasso: A Collaborative Hierarchical Sparse Modeling Framework

Abstract

Sparse modeling is a powerful framework for data analysis and processing. Traditionally, encoding in this framework is performed by solving an L1-regularized linear regression problem, commonly referred to as Lasso or Basis Pursuit. In this work we combine the sparsity-inducing property of the Lasso model at the individual feature level, with the block-sparsity property of the Group Lasso model, where sparse groups of features are jointly encoded, obtaining a sparsity pattern hierarchically structured. This results in the Hierarchical Lasso (HiLasso), which shows important practical modeling advantages. We then extend this approach to the collaborative case, where a set of simultaneously coded signals share the same sparsity pattern at the higher (group) level, but not necessarily at the lower (inside the group) level, obtaining the collaborative HiLasso model (C-HiLasso). Such signals then share the same active groups, or classes, but not necessarily the same active set. This model is very well suited for applications such as source identification and separation. An efficient optimization procedure, which guarantees convergence to the global optimum, is developed for these new models. The underlying presentation of the new framework and optimization approach is complemented with experimental examples and theoretical results regarding recovery guarantees for the proposed models.

References in corpus (5)

• Reduce and Boost: Recovering Arbitrary Sets of Jointly Sparse Vectors
• Proximal Methods for Hierarchical Sparse Coding
• A direct formulation for sparse PCA using semidefinite programming
• Block-length dependent thresholds in block-sparse compressed sensing
• Sensing Matrix Optimization for Block-Sparse Decoding

Cited by in corpus (36)

• Structured Compressed Sensing: From Theory to Applications
• A convex model for non-negative matrix factorization and dimensionality reduction on physical space
• Multimodal Task-Driven Dictionary Learning for Image Classification
• Structured Priors for Sparse-Representation-Based Hyperspectral Image Classification
• In-network Sparsity-regularized Rank Minimization: Algorithms and Applications
• Task-Driven Dictionary Learning for Hyperspectral Image Classification with Structured Sparsity Constraints
• The Landmark Selection Method for Multiple Output Prediction
• Nested Sparse Approximation: Structured Estimation of V2V Channels Using Geometry-Based Stochastic Channel Model
• ICR: Iterative Convex Refinement for Sparse Signal Recovery Using Spike and Slab Priors
• Sparsity-based Color Image Super Resolution via Exploiting Cross Channel Constraints
• GAP Safe Screening Rules for Sparse-Group-Lasso
• Forest Sparsity for Multi-channel Compressive Sensing
• Reliable recovery of hierarchically sparse signals for Gaussian and Kronecker product measurements
• Exploiting Low-dimensional Structures to Enhance DNN Based Acoustic Modeling in Speech Recognition
• Structured Dictionary Learning for Classification
• Learning Efficient Structured Sparse Models
• Spatio-Temporal Structured Sparse Regression with Hierarchical Gaussian Process Priors
• Simultaneously Structured Models with Application to Sparse and Low-rank Matrices
• Learning efficient sparse and low rank models
• Two-Layer Feature Reduction for Sparse-Group Lasso via Decomposition of Convex Sets
• Block Sparse Multi-lead ECG Compression Exploiting between-lead Collaboration
• Semi-device-dependent blind quantum tomography
• Performance of Hierarchical Sparse Detectors for Massive MTC
• Stock Market Prediction from WSJ: Text Mining via Sparse Matrix Factorization
• Hierarchical Isometry Properties of Hierarchical Measurements
• Classification with Sparse Overlapping Groups
• Hierarchical Sparse and Collaborative Low-Rank Representation for Emotion Recognition
• Collaborative Multi-sensor Classification via Sparsity-based Representation
• Fast Hyperspectral Unmixing in Presence of Nonlinearity or Mismodelling Effects
• New explicit thresholding/shrinkage formulas for one class of regularization problems with overlapping group sparsity and their applications
• Hierarchical sparse recovery from hierarchically structured measurements with application to massive random access
• Discriminative models for robust image classification
• Hyperspectral Unmixing Overview: Geometrical, Statistical, and Sparse Regression-Based Approaches
• Adaptive matching pursuit for sparse signal recovery
• Guaranteed blind deconvolution and demixing via hierarchically sparse reconstruction
• One-Shot Messaging at Any Load Through Random Sub-Channeling in OFDM