A Fast and Scalable Polyatomic Frank-Wolfe Algorithm for the LASSO

Abstract

We propose a fast and scalable Polyatomic Frank-Wolfe (P-FW) algorithm for the resolution of high-dimensional LASSO regression problems. The latter improves upon traditional Frank-Wolfe methods by considering generalized greedy steps with polyatomic (i.e. linear combinations of multiple atoms) update directions, hence allowing for a more efficient exploration of the search space. To preserve sparsity of the intermediate iterates, we re-optimize the LASSO problem over the set of selected atoms at each iteration. For efficiency reasons, the accuracy of this re-optimization step is relatively low for early iterations and gradually increases with the iteration count. We provide convergence guarantees for our algorithm and validate it in simulated compressed sensing setups. Our experiments reveal that P-FW outperforms state-of-the-art methods in terms of runtime, both for FW methods and optimal first-order proximal gradient methods such as the Fast Iterative Soft-Thresholding Algorithm (FISTA).

arXiv commentPublished in IEEE Signal Processing Letters

References in corpus (20)

• Rejoinder to "Least angle regression" by Efron et al
• Least Angle Regression
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Pathwise coordinate optimization
• Image Reconstruction: From Sparsity to Data-adaptive Methods and Machine Learning
• Compressed sensing imaging techniques for radio interferometry
• The Sliding Frank-Wolfe Algorithm and its Application to Super-Resolution Microscopy
• Splines are Universal Solutions of Linear Inverse Problems with Generalized-TV regularization
• Sparsity of solutions for variational inverse problems with finite-dimensional data
• Continuous-Domain Solutions of Linear Inverse Problems with Tikhonov vs. Generalized TV Regularization
• TV-based Reconstruction of Periodic Functions
• Periodic Splines and Gaussian Processes for the Resolution of Linear Inverse Problems
• Functional Penalised Basis Pursuit on Spheres

Cited by in corpus (1)

• PolyCLEAN: Atomic Optimization for Super-Resolution Imaging and Uncertainty Estimation in Radio Interferometry