Fixed Point Strategies in Data Science

Abstract

The goal of this paper is to promote the use of fixed point strategies in data science by showing that they provide a simplifying and unifying framework to model, analyze, and solve a great variety of problems. They are seen to constitute a natural environment to explain the behavior of advanced convex optimization methods as well as of recent nonlinear methods in data science which are formulated in terms of paradigms that go beyond minimization concepts and involve constructs such as Nash equilibria or monotone inclusions. We review the pertinent tools of fixed point theory and describe the main state-of-the-art algorithms for provably convergent fixed point construction. We also incorporate additional ingredients such as stochasticity, block-implementations, and non-Euclidean metrics, which provide further enhancements. Applications to signal and image processing, machine learning, statistics, neural networks, and inverse problems are discussed.

References in corpus (10)

• SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives
• Safe Feature Elimination in Sparse Supervised Learning
• Plug-and-Play Methods Provably Converge with Properly Trained Denoisers
• Finito: A Faster, Permutable Incremental Gradient Method for Big Data Problems
• Provable Convergence of Plug-and-Play Priors with MMSE denoisers
• A Survey of Game Theoretic Approaches for Adversarial Machine Learning in Cybersecurity Tasks
• Lipschitz constant estimation of Neural Networks via sparse polynomial optimization
• A Warped Resolvent Algorithm to Construct Nash Equilibria
• A Hierarchical Convex Optimization for Multiclass SVM Achieving Maximum Pairwise Margins with Least Empirical Hinge-Loss
• Sparse Prediction with the $k$-Support Norm

Cited by in corpus (12)

• Graphical Inference in Linear-Gaussian State-Space Models
• Federated Learning Using Three-Operator ADMM
• The difference vectors for convex sets and a resolution of the geometry conjecture
• Attouch-Théra Duality, Generalized Cycles and Gap Vectors
• A Warped Resolvent Algorithm to Construct Nash Equilibria
• From Noisy Fixed-Point Iterations to Private ADMM for Centralized and Federated Learning
• From Contraction Theory to Fixed Point Algorithms on Riemannian and Non-Euclidean Spaces
• Robust Implicit Networks via Non-Euclidean Contractions
• Regularization of Inverse Problems: Deep Equilibrium Models versus Bilevel Learning
• A Proximal Variable Smoothing for Nonsmooth Minimization Involving Weakly Convex Composite with MIMO Application
• WaveNeXt 2: ConvNeXt-Based Fast Neural Vocoders With Residual Denoising and Sub-Modeling for GAN and Diffusion Models
• Analysis and Numerical Solution of a Modular Convex Nash Equilibrium Problem