Learning step sizes for unfolded sparse coding
arXiv:1905.11071
Abstract
Sparse coding is typically solved by iterative optimization techniques, such as the Iterative Shrinkage-Thresholding Algorithm (ISTA). Unfolding and learning weights of ISTA using neural networks is a practical way to accelerate estimation. In this paper, we study the selection of adapted step sizes for ISTA. We show that a simple step size strategy can improve the convergence rate of ISTA by leveraging the sparsity of the iterates. However, it is impractical in most large-scale applications. Therefore, we propose a network architecture where only the step sizes of ISTA are learned. We demonstrate that for a large class of unfolded algorithms, if the algorithm converges to the solution of the Lasso, its last layers correspond to ISTA with learned step sizes. Experiments show that our method is competitive with state-of-the-art networks when the solutions are sparse enough.
22 pages
References in corpus (19)
- Scikit-learn: Machine Learning in Python
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Rejoinder to "Least angle regression" by Efron et al
- Least Angle Regression
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Pathwise coordinate optimization
- AMP-Inspired Deep Networks for Sparse Linear Inverse Problems
- Deep Unfolding: Model-Based Inspiration of Novel Deep Architectures
- Safe Feature Elimination in Sparse Supervised Learning
- Theoretical Linear Convergence of Unfolded ISTA and its Practical Weights and Thresholds
- Maximal Sparsity with Deep Networks?
- Local Linear Convergence of Forward-Backward under Partial Smoothness
- Learning to solve inverse problems using Wasserstein loss