A Field Guide to Forward-Backward Splitting with a FASTA Implementation

Abstract

Non-differentiable and constrained optimization play a key role in machine learning, signal and image processing, communications, and beyond. For high-dimensional minimization problems involving large datasets or many unknowns, the forward-backward splitting method provides a simple, practical solver. Despite its apparently simplicity, the performance of the forward-backward splitting is highly sensitive to implementation details. This article is an introductory review of forward-backward splitting with a special emphasis on practical implementation concerns. Issues like stepsize selection, acceleration, stopping conditions, and initialization are considered. Numerical experiments are used to compare the effectiveness of different approaches. Many variations of forward-backward splitting are implemented in the solver FASTA (short for Fast Adaptive Shrinkage/Thresholding Algorithm). FASTA provides a simple interface for applying forward-backward splitting to a broad range of problems.

References in corpus (4)

• Algorithms, Initializations, and Convergence for the Nonnegative Matrix Factorization
• Adaptive Primal-Dual Hybrid Gradient Methods for Saddle-Point Problems
• Low-rank Matrix Completion with Noisy Observations: a Quantitative Comparison
• Anti-sparse coding for approximate nearest neighbor search

Cited by in corpus (36)

• A Survey on Data Augmentation for Text Classification
• On the use of deep learning for phase recovery
• Transferable Clean-Label Poisoning Attacks on Deep Neural Nets
• Regularized Inverse Holographic Volume Reconstruction for 3D Particle Tracking
• Adaptive ADMM with Spectral Penalty Parameter Selection
• Reliable OFDM Receiver with Ultra-Low Resolution ADC
• Bilinear Recovery using Adaptive Vector-AMP
• Towards Optimal Structured CNN Pruning via Generative Adversarial Learning
• Mitigating Smart Jammers in Multi-User MIMO
• Phase Retrieval via Polytope Optimization: Geometry, Phase Transitions, and New Algorithms
• prDeep: Robust Phase Retrieval with a Flexible Deep Network
• Adversarial attacks on Copyright Detection Systems
• A continuous adjoint for photo-acoustic tomography of the brain
• Multimode fiber based single-shot full-field measurement of optical pulses
• Towards Compact ConvNets via Structure-Sparsity Regularized Filter Pruning
• Blur resolved OCT: full-range interferometric synthetic aperture microscopy through dispersion encoding
• Spectral Frank-Wolfe Algorithm: Strict Complementarity and Linear Convergence
• Fast Low-Rank Matrix Estimation without the Condition Number
• Unwrapping ADMM: Efficient Distributed Computing via Transpose Reduction
• Improved Algorithms for Matrix Recovery from Rank-One Projections
• On phase retrieval via matrix completion and the estimation of low rank PSD matrices
• Fundamental Limits of PhaseMax for Phase Retrieval: A Replica Analysis
• Active Learning Under Malicious Mislabeling and Poisoning Attacks
• Privacy-Preserving Federated Learning on Partitioned Attributes
• Regularization by Denoising: Clarifications and New Interpretations
• Gradient flows and randomised thresholding: sparse inversion and classification
• ${\ell }^p-{\ell }^q$-Norm Minimization for Joint Precoding and Peak-to-Average-Power Ratio Reduction
• Being Single Has Benefits. Instance Poisoning to Deceive Malware Classifiers
• Across-domains transferability of Deep-RED in de-noising and compressive sensing recovery of seismic data
• Finite-Alphabet MMSE Equalization for All-Digital Massive MU-MIMO mmWave Communication
• 1-bit Massive MU-MIMO Precoding in VLSI
• $k$FW: A Frank-Wolfe style algorithm with stronger subproblem oracles
• Soft-Output Finite Alphabet Equalization for mmWAVE Massive MIMO
• Finite-Alphabet Wiener Filter Precoding for mmWave Massive MU-MIMO Systems
• Soft-Output Joint Channel Estimation and Data Detection using Deep Unfolding
• Graph- and finite element-based total variation models for the inverse problem in diffuse optical tomography