Learning Efficient Structured Sparse Models

Abstract

We present a comprehensive framework for structured sparse coding and modeling extending the recent ideas of using learnable fast regressors to approximate exact sparse codes. For this purpose, we develop a novel block-coordinate proximal splitting method for the iterative solution of hierarchical sparse coding problems, and show an efficient feed forward architecture derived from its iteration. This architecture faithfully approximates the exact structured sparse codes with a fraction of the complexity of the standard optimization methods. We also show that by using different training objective functions, learnable sparse encoders are no longer restricted to be mere approximants of the exact sparse code for a pre-given dictionary, as in earlier formulations, but can be rather used as full-featured sparse encoders or even modelers. A simple implementation shows several orders of magnitude speedup compared to the state-of-the-art at minimal performance degradation, making the proposed framework suitable for real time and large-scale applications.

ICML2012

References in corpus (5)

• The composite absolute penalties family for grouped and hierarchical variable selection
• A note on the group lasso and a sparse group lasso
• Proximal Methods for Hierarchical Sparse Coding
• Fast Inference in Sparse Coding Algorithms with Applications to Object Recognition
• C-HiLasso: A Collaborative Hierarchical Sparse Modeling Framework

Cited by in corpus (6)

• AMP-Inspired Deep Networks for Sparse Linear Inverse Problems
• Learning to Optimize: A Primer and A Benchmark
• Learning optimal nonlinearities for iterative thresholding algorithms
• On the Convergence of Learning-based Iterative Methods for Nonconvex Inverse Problems
• Convolutional Neural Networks Analyzed via Inverse Problem Theory and Sparse Representations
• Learning to solve TV regularized problems with unrolled algorithms