${\ell }_0$ Sparsifying Transform Learning with Efficient Optimal Updates and Convergence Guarantees

Abstract

Many applications in signal processing benefit from the sparsity of signals in a certain transform domain or dictionary. Synthesis sparsifying dictionaries that are directly adapted to data have been popular in applications such as image denoising, inpainting, and medical image reconstruction. In this work, we focus instead on the sparsifying transform model, and study the learning of well-conditioned square sparsifying transforms. The proposed algorithms alternate between a ${\ell }_0$ "norm"-based sparse coding step, and a non-convex transform update step. We derive the exact analytical solution for each of these steps. The proposed solution for the transform update step achieves the global minimum in that step, and also provides speedups over iterative solutions involving conjugate gradients. We establish that our alternating algorithms are globally convergent to the set of local minimizers of the non-convex transform learning problems. In practice, the algorithms are insensitive to initialization. We present results illustrating the promising performance and significant speed-ups of transform learning over synthesis K-SVD in image denoising.

Accepted to IEEE Transactions on Signal Processing

References in corpus (1)

• Constrained Overcomplete Analysis Operator Learning for Cosparse Signal Modelling

Cited by in corpus (20)

• MoDL: Model Based Deep Learning Architecture for Inverse Problems
• PWLS-ULTRA: An Efficient Clustering and Learning-Based Approach for Low-Dose 3D CT Image Reconstruction
• Graph topology inference based on sparsifying transform learning
• Data-Driven Learning of a Union of Sparsifying Transforms Model for Blind Compressed Sensing
• Convolutional Analysis Operator Learning: Acceleration and Convergence
• SPULTRA: Low-Dose CT Image Reconstruction with Joint Statistical and Learned Image Models
• Low Dose CT Image Reconstruction With Learned Sparsifying Transform
• Learning Fast Sparsifying Transforms
• Convolutional Analysis Operator Learning: Dependence on Training Data
• DeConFuse : A Deep Convolutional Transform based Unsupervised Fusion Framework
• Complete Dictionary Learning via ${\ell }^4$-Norm Maximization over the Orthogonal Group
• A Set-Theoretic Study of the Relationships of Image Models and Priors for Restoration Problems
• DECT-MULTRA: Dual-Energy CT Image Decomposition With Learned Mixed Material Models and Efficient Clustering
• Learning to compress and search visual data in large-scale systems
• Multi-layer Residual Sparsifying Transform (MARS) Model for Low-dose CT Image Reconstruction
• Two-layer clustering-based sparsifying transform learning for low-dose CT reconstruction
• Reconciliation of Statistical and Spatial Sparsity For Robust Image and Image-Set Classification
• Blind Denoising Autoencoder
• Two-layer Residual Sparsifying Transform Learning for Image Reconstruction
• Multiscale Sparsifying Transform Learning for Image Denoising