Convolutional Analysis Operator Learning: Acceleration and Convergence

Abstract

Convolutional operator learning is gaining attention in many signal processing and computer vision applications. Learning kernels has mostly relied on so-called patch-domain approaches that extract and store many overlapping patches across training signals. Due to memory demands, patch-domain methods have limitations when learning kernels from large datasets -- particularly with multi-layered structures, e.g., convolutional neural networks -- or when applying the learned kernels to high-dimensional signal recovery problems. The so-called convolution approach does not store many overlapping patches, and thus overcomes the memory problems particularly with careful algorithmic designs; it has been studied within the "synthesis" signal model, e.g., convolutional dictionary learning. This paper proposes a new convolutional analysis operator learning (CAOL) framework that learns an analysis sparsifying regularizer with the convolution perspective, and develops a new convergent Block Proximal Extrapolated Gradient method using a Majorizer (BPEG-M) to solve the corresponding block multi-nonconvex problems. To learn diverse filters within the CAOL framework, this paper introduces an orthogonality constraint that enforces a tight-frame filter condition, and a regularizer that promotes diversity between filters. Numerical experiments show that, with sharp majorizers, BPEG-M significantly accelerates the CAOL convergence rate compared to the state-of-the-art block proximal gradient (BPG) method. Numerical experiments for sparse-view computational tomography show that a convolutional sparsifying regularizer learned via CAOL significantly improves reconstruction quality compared to a conventional edge-preserving regularizer. Using more and wider kernels in a learned regularizer better preserves edges in reconstructed images.

22 pages, 11 figures, fixed incorrect math theorem numbers in fig. 3

References in corpus (8)

• Convolutional Dictionary Learning: Acceleration and Convergence
• Deep BCD-Net Using Identical Encoding-Decoding CNN Structures for Iterative Image Recovery
• ${\ell }_0$ Sparsifying Transform Learning with Efficient Optimal Updates and Convergence Guarantees
• Compressed Sensing and Parallel Acquisition
• Convolutional Analysis Operator Learning: Acceleration and Convergence
• Online Convolutional Dictionary Learning
• Uniform Recovery from Subgaussian Multi-Sensor Measurements
• Convolutional Analysis Operator Learning: Dependence on Training Data

Cited by in corpus (8)

• Convolutional Analysis Operator Learning: Acceleration and Convergence
• Bilevel methods for image reconstruction
• HePPCAT: Probabilistic PCA for Data with Heteroscedastic Noise
• Learning Deep Analysis Dictionaries for Image Super-Resolution
• Momentum-Net: Fast and convergent iterative neural network for inverse problems
• Momentum-Net for Low-Dose CT Image Reconstruction
• Provably Convergent Learned Inexact Descent Algorithm for Low-Dose CT Reconstruction
• Deep learning based dictionary learning and tomographic image reconstruction