Task-Driven Dictionary Learning

Abstract

Modeling data with linear combinations of a few elements from a learned dictionary has been the focus of much recent research in machine learning, neuroscience and signal processing. For signals such as natural images that admit such sparse representations, it is now well established that these models are well suited to restoration tasks. In this context, learning the dictionary amounts to solving a large-scale matrix factorization problem, which can be done efficiently with classical optimization tools. The same approach has also been used for learning features from data for other purposes, e.g., image classification, but tuning the dictionary in a supervised way for these tasks has proven to be more difficult. In this paper, we present a general formulation for supervised dictionary learning adapted to a wide variety of tasks, and present an efficient algorithm for solving the corresponding optimization problem. Experiments on handwritten digit classification, digital art identification, nonlinear inverse image problems, and compressed sensing demonstrate that our approach is effective in large-scale settings, and is well suited to supervised and semi-supervised classification, as well as regression tasks for data that admit sparse representations.

final draft post-refereeing

References in corpus (2)

• Supervised Topic Models
• Supervised Dictionary Learning

Cited by in corpus (109)

• Deep Unfolding: Model-Based Inspiration of Novel Deep Architectures
• Group Component Analysis for Multiblock Data: Common and Individual Feature Extraction
• Multimodal Task-Driven Dictionary Learning for Image Classification
• OptNet: Differentiable Optimization as a Layer in Neural Networks
• Non-Intrusive Energy Disaggregation Using Non-negative Matrix Factorization with Sum-to-k Constraint
• Cloud K-SVD: A Collaborative Dictionary Learning Algorithm for Big, Distributed Data
• Intelligent Meta-Imagers: From Compressed to Learned Sensing
• Super-Resolution with Deep Convolutional Sufficient Statistics
• Linearized Kernel Dictionary Learning
• Kernelized Supervised Dictionary Learning
• Deep Dictionary Learning: A PARametric NETwork Approach
• Multiple Kernel Sparse Representations for Supervised and Unsupervised Learning
• On the Identifiability of Overcomplete Dictionaries via the Minimisation Principle Underlying K-SVD
• Decentralized Online Learning with Kernels
• Task-Driven Dictionary Learning for Hyperspectral Image Classification with Structured Sparsity Constraints
• Denoising of gravitational wave signals via dictionary learning algorithms
• First and Second Order Methods for Online Convolutional Dictionary Learning
• Efficient and Modular Implicit Differentiation
• Convolutional Neural Networks Analyzed via Convolutional Sparse Coding
• Supervised Dictionary Learning and Sparse Representation-A Review
• Shapelet-based Sparse Representation for Landcover Classification of Hyperspectral Images
• Exploiting Restricted Boltzmann Machines and Deep Belief Networks in Compressed Sensing
• Bilevel methods for image reconstruction
• On the Adversarial Robustness of LASSO Based Feature Selection
• Discriminative Recurrent Sparse Auto-Encoders
• Infinite-dimensional inverse problems with finite measurements
• Implicit differentiation of Lasso-type models for hyperparameter optimization
• Minimax Lower Bounds on Dictionary Learning for Tensor Data
• Learning the nonlinear geometry of high-dimensional data: Models and algorithms
• Structured Dictionary Learning for Classification
• Structure-Aware Classification using Supervised Dictionary Learning
• Robust Non-Linear Matrix Factorization for Dictionary Learning, Denoising, and Clustering
• Fault-Tolerant Control of Linear Quantum Stochastic Systems
• Learning Fast Sparsifying Transforms
• Learning Deep ${\ell }_0$ Encoders
• Dictionary Learning from Incomplete Data
• A fast patch-dictionary method for whole image recovery
• Patchwise Joint Sparse Tracking with Occlusion Detection
• Weakly-supervised Dictionary Learning
• Discriminative Nonlinear Analysis Operator Learning: When Cosparse Model Meets Image Classification
• Group Sparsity Residual Constraint for Image Denoising
• Supervised Learning of Sparsity-Promoting Regularizers for Denoising
• Generative discriminative models for multivariate inference and statistical mapping in medical imaging
• Fast and Robust Archetypal Analysis for Representation Learning
• Learning A Task-Specific Deep Architecture For Clustering
• Complete Dictionary Learning via ${\ell }^4$-Norm Maximization over the Orthogonal Group
• Matrix Cofactorization for Joint Representation Learning and Supervised Classification -- Application to Hyperspectral Image Analysis
• Deep Network Classification by Scattering and Homotopy Dictionary Learning
• On the Sample Complexity of Predictive Sparse Coding
• Adversarial Robustness of Supervised Sparse Coding
• Automatic Target Recognition on Synthetic Aperture Radar Imagery: A Survey
• Confident Kernel Sparse Coding and Dictionary Learning
• Travel time tomography with adaptive dictionaries
• Good Similar Patches for Image Denoising
• Efficient Sparse Coding using Hierarchical Riemannian Pursuit
• Sparse Linear Regression With Missing Data
• Implicitly Defined Layers in Neural Networks
• Identifiability of Complete Dictionary Learning
• Jointly Learning Non-negative Projection and Dictionary with Discriminative Graph Constraints for Classification
• A Picture is Worth a Billion Bits: Real-Time Image Reconstruction from Dense Binary Pixels
• Efficient Dictionary Learning via Very Sparse Random Projections
• Online Discriminative Dictionary Learning for Image Classification Based on Block-Coordinate Descent Method
• An efficient supervised dictionary learning method for audio signal recognition
• Multimodal Sparse Bayesian Dictionary Learning
• Extrinsic Methods for Coding and Dictionary Learning on Grassmann Manifolds
• Computational Cost Reduction in Learned Transform Classifications
• Joint Subspace Recovery and Enhanced Locality Driven Robust Flexible Discriminative Dictionary Learning
• Cloud K-SVD for Image Denoising
• Dictionary Learning with Almost Sure Error Constraints
• Decentralized Dynamic Discriminative Dictionary Learning
• Improved Estimation in Time Varying Models
• A Theory of Feature Learning
• Sparse Factorization Layers for Neural Networks with Limited Supervision
• Super-efficiency of automatic differentiation for functions defined as a minimum
• Evolutionary Simplicial Learning as a Generative and Compact Sparse Framework for Classification
• Amortized Implicit Differentiation for Stochastic Bilevel Optimization
• On the convergence of group-sparse autoencoders
• Automatic Cross-Domain Transfer Learning for Linear Regression
• Fully Trainable and Interpretable Non-Local Sparse Models for Image Restoration
• Learning Simple Thresholded Features with Sparse Support Recovery
• Supervised Deep Sparse Coding Networks
• Multiple Instance Dictionary Learning using Functions of Multiple Instances
• Designing A Composite Dictionary Adaptively From Joint Examples
• Template matching with noisy patches: A contrast-invariant GLR test
• Dictionary-Learning-Based Data Pruning for System Identification
• Probabilistic Modelling of Signal Mixtures with Differentiable Dictionaries
• Data-driven audio recognition: a supervised dictionary approach
• A Probabilistic Framework for Discriminative Dictionary Learning
• Proximal Mapping for Deep Regularization
• Sparse and redundant signal representations for x-ray computed tomography
• Spatial-Aware Dictionary Learning for Hyperspectral Image Classification
• Learning Hybrid Representation by Robust Dictionary Learning in Factorized Compressed Space
• Information-theoretic Dictionary Learning for Image Classification
• On the Invariance of Dictionary Learning and Sparse Representation to Projecting Data to a Discriminative Space
• Machine olfaction using time scattering of sensor multiresolution graphs
• Multiple Instance Dictionary Learning for Beat-to-Beat Heart Rate Monitoring from Ballistocardiograms
• Dictionary Learning for Two-Dimensional Kendall Shapes
• Discriminative reconstruction via simultaneous dense and sparse coding
• Sparse Coding with Fast Image Alignment via Large Displacement Optical Flow
• Conditional Sparse Coding and Grouped Multivariate Regression
• Eigen component analysis: A quantum theory incorporated machine learning technique to find linearly maximum separable components
• Learning Stable Multilevel Dictionaries for Sparse Representations
• Joint Learning of Discriminative Low-dimensional Image Representations Based on Dictionary Learning and Two-layer Orthogonal Projections
• Sparse Coding Driven Deep Decision Tree Ensembles for Nuclear Segmentation in Digital Pathology Images
• Jointly Learning Structured Analysis Discriminative Dictionary and Analysis Multiclass Classifier
• Local Similarities, Global Coding: An Algorithm for Feature Coding and its Applications
• Deep Symbolic Representation Learning for Heterogeneous Time-series Classification
• Multi-Scale Saliency Detection using Dictionary Learning
• Knowledge Distillation By Sparse Representation Matching