Fixed Point Strategies in Data Science

Abstract

The goal of this paper is to promote the use of fixed point strategies in data science by showing that they provide a simplifying and unifying framework to model, analyze, and solve a great variety of problems. They are seen to constitute a natural environment to explain the behavior of advanced convex optimization methods as well as of recent nonlinear methods in data science which are formulated in terms of paradigms that go beyond minimization concepts and involve constructs such as Nash equilibria or monotone inclusions. We review the pertinent tools of fixed point theory and describe the main state-of-the-art algorithms for provably convergent fixed point construction. We also incorporate additional ingredients such as stochasticity, block-implementations, and non-Euclidean metrics, which provide further enhancements. Applications to signal and image processing, machine learning, statistics, neural networks, and inverse problems are discussed.

References in corpus (40)

• Intriguing properties of neural networks
• Fast Image Recovery Using Variable Splitting and Constrained Optimization
• An Augmented Lagrangian Approach to the Constrained Optimization Formulation of Imaging Inverse Problems
• SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives
• BM3D Frames and Variational Image Deblurring
• Restoration of Poissonian Images Using Alternating Direction Optimization
• Generalized Forward-Backward Splitting
• A distributed primal-dual algorithm for computation of generalized Nash equilibria with shared affine coupling constraints via operator splitting methods
• Proximal Methods for Hierarchical Sparse Coding
• An Online Plug-and-Play Algorithm for Regularized Image Reconstruction
• Optimal Transport with Proximal Splitting
• Safe Feature Elimination in Sparse Supervised Learning
• Distributed generalized Nash equilibrium seeking in aggregative games on time-varying networks
• Lipschitz regularity of deep neural networks: analysis and efficient estimation
• Proximal Splitting Methods in Signal Processing
• On a generalization of the iterative soft-thresholding algorithm for the case of non-separable penalty
• Plug-and-Play Methods Provably Converge with Properly Trained Denoisers
• Distributed generalized Nash equilibria computation of monotone games via a preconditioned proximal point algorithm
• Finito: A Faster, Permutable Incremental Gradient Method for Big Data Problems
• A Non-Local Structure Tensor Based Approach for Multicomponent Image Recovery Problems
• Positive deconvolution for superimposed extended source and point sources
• Provable Convergence of Plug-and-Play Priors with MMSE denoisers
• A Survey of Game Theoretic Approaches for Adversarial Machine Learning in Cybersecurity Tasks
• Parseval Proximal Neural Networks
• Algorithms and Convergence Results of Projection Methods for Inconsistent Feasibility Problems: A Review
• On perturbed proximal gradient algorithms
• Linearly Involved Generalized Moreau Enhanced Models and Their Proximal Splitting Algorithm under Overall Convexity Condition
• A Primal-Dual Proximal Algorithm for Sparse Template-Based Adaptive Filtering: Application to Seismic Multiple Removal
• Bandit learning in concave $N$-person games
• Lipschitz constant estimation of Neural Networks via sparse polynomial optimization
• Stochastic Approximations and Perturbations in Forward-Backward Splitting for Monotone Operators
• A Random Block-Coordinate Douglas-Rachford Splitting Method with Low Computational Complexity for Binary Logistic Regression
• Data-driven nonsmooth optimization
• A Proximal Approach for a Class of Matrix Optimization Problems
• On Decomposition Models in Imaging Sciences and Multi-time Hamilton-Jacobi Partial Differential Equations
• Quasinonexpansive Iterations on the Affine Hull of Orbits: From Mann's Mean Value Algorithm to Inertial Methods
• Deep Transform and Metric Learning Network: Wedding Deep Dictionary Learning and Neural Networks
• A Warped Resolvent Algorithm to Construct Nash Equilibria
• A Hierarchical Convex Optimization for Multiclass SVM Achieving Maximum Pairwise Margins with Least Empirical Hinge-Loss
• Sparse Prediction with the $k$-Support Norm

Cited by in corpus (13)

• Dualize, Split, Randomize: Toward Fast Nonsmooth Optimization Algorithms
• Graphical Inference in Linear-Gaussian State-Space Models
• Federated Learning Using Three-Operator ADMM
• The difference vectors for convex sets and a resolution of the geometry conjecture
• Attouch-Théra Duality, Generalized Cycles and Gap Vectors
• A Warped Resolvent Algorithm to Construct Nash Equilibria
• From Noisy Fixed-Point Iterations to Private ADMM for Centralized and Federated Learning
• Regularization of Inverse Problems: Deep Equilibrium Models versus Bilevel Learning
• From Contraction Theory to Fixed Point Algorithms on Riemannian and Non-Euclidean Spaces
• Robust Implicit Networks via Non-Euclidean Contractions
• A Proximal Variable Smoothing for Nonsmooth Minimization Involving Weakly Convex Composite with MIMO Application
• Analysis and Numerical Solution of a Modular Convex Nash Equilibrium Problem
• WaveNeXt 2: ConvNeXt-Based Fast Neural Vocoders With Residual Denoising and Sub-Modeling for GAN and Diffusion Models