First and Second Order Methods for Online Convolutional Dictionary Learning

Abstract

Convolutional sparse representations are a form of sparse representation with a structured, translation invariant dictionary. Most convolutional dictionary learning algorithms to date operate in batch mode, requiring simultaneous access to all training images during the learning process, which results in very high memory usage and severely limits the training data that can be used. Very recently, however, a number of authors have considered the design of online convolutional dictionary learning algorithms that offer far better scaling of memory and computational cost with training set size than batch methods. This paper extends our prior work, improving a number of aspects of our previous algorithm; proposing an entirely new one, with better performance, and that supports the inclusion of a spatial mask for learning from incomplete data; and providing a rigorous theoretical analysis of these methods.

References in corpus (5)

• A survey of sparse representation: algorithms and applications
• Spectral Representations for Convolutional Neural Networks
• Working Locally Thinking Globally: Theoretical Guarantees for Convolutional Sparse Coding
• Multi-Layer Convolutional Sparse Modeling: Pursuit and Dictionary Learning
• ADMM Penalty Parameter Selection by Residual Balancing

Cited by in corpus (7)

• Convolutional Dictionary Learning: A Comparative Review and New Algorithms
• Multi-Layer Convolutional Sparse Modeling: Pursuit and Dictionary Learning
• Learning Convolutional Sparse Coding on Complex Domain for Interferometric Phase Restoration
• Group Invariant Dictionary Learning
• Short-and-Sparse Deconvolution -- A Geometric Approach
• Convolutional Dictionary Regularizers for Tomographic Inversion
• Tensor Convolutional Sparse Coding with Low-Rank activations, an application to EEG analysis