Convolutional Dictionary Learning: Acceleration and Convergence

Abstract

Convolutional dictionary learning (CDL or sparsifying CDL) has many applications in image processing and computer vision. There has been growing interest in developing efficient algorithms for CDL, mostly relying on the augmented Lagrangian (AL) method or the variant alternating direction method of multipliers (ADMM). When their parameters are properly tuned, AL methods have shown fast convergence in CDL. However, the parameter tuning process is not trivial due to its data dependence and, in practice, the convergence of AL methods depends on the AL parameters for nonconvex CDL problems. To moderate these problems, this paper proposes a new practically feasible and convergent Block Proximal Gradient method using a Majorizer (BPG-M) for CDL. The BPG-M-based CDL is investigated with different block updating schemes and majorization matrix designs, and further accelerated by incorporating some momentum coefficient formulas and restarting techniques. All of the methods investigated incorporate a boundary artifacts removal (or, more generally, sampling) operator in the learning model. Numerical experiments show that, without needing any parameter tuning process, the proposed BPG-M approach converges more stably to desirable solutions of lower objective values than the existing state-of-the-art ADMM algorithm and its memory-efficient variant do. Compared to the ADMM approaches, the BPG-M method using a multi-block updating scheme is particularly useful in single-threaded CDL algorithm handling large datasets, due to its lower memory requirement and no polynomial computational complexity. Image denoising experiments show that, for relatively strong additive white Gaussian noise, the filters learned by BPG-M-based CDL outperform those trained by the ADMM approach.

21 pages, 7 figures, submitted to IEEE Transactions on Image Processing

References in corpus (5)

• Convolutional Sparse Coding for High Dynamic Range Imaging
• Efficient Sum of Outer Products Dictionary Learning (SOUP-DIL) and Its Application to Inverse Problems
• Optimization Methods for Convolutional Sparse Coding
• Convolutional Neural Networks Analyzed via Convolutional Sparse Coding
• Working Locally Thinking Globally - Part II: Stability and Algorithms for Convolutional Sparse Coding

Cited by in corpus (21)

• Image Reconstruction: From Sparsity to Data-adaptive Methods and Machine Learning
• Convolutional Dictionary Learning: A Comparative Review and New Algorithms
• Deep BCD-Net Using Identical Encoding-Decoding CNN Structures for Iterative Image Recovery
• Convolutional Analysis Operator Learning: Acceleration and Convergence
• Bilevel methods for image reconstruction
• Systematic Review on Learning-based Spectral CT
• Momentum-Net: Fast and convergent iterative neural network for inverse problems
• Optimization methods for MR image reconstruction (long version)
• Convolutional Analysis Operator Learning: Dependence on Training Data
• Model-based multi-parameter mapping
• Quantitative MR Image Reconstruction using Parameter-Specific Dictionary Learning with Adaptive Dictionary-Size and Sparsity-Level Choice
• Momentum-Net for Low-Dose CT Image Reconstruction
• Short-and-Sparse Deconvolution -- A Geometric Approach
• Multi-Channel Convolutional Analysis Operator Learning for Dual-Energy CT Reconstruction
• A Tool for Super-Resolving Multimodal Clinical MRI
• Convolutional Dictionary Regularizers for Tomographic Inversion
• Convolutional Dictionary Pair Learning Network for Image Representation Learning
• On the Reconstruction Risk of Convolutional Sparse Dictionary Learning
• Provably Convergent Learned Inexact Descent Algorithm for Low-Dose CT Reconstruction
• Learning Deep Analysis Dictionaries -- Part II: Convolutional Dictionaries
• Deep Micro-Dictionary Learning and Coding Network