A Stochastic forward-backward splitting method for solving monotone inclusions in Hilbert spaces

Abstract

We propose and analyze the convergence of a novel stochastic forward-backward splitting algorithm for solving monotone inclusions given by the sum of a maximal monotone operator and a single-valued maximal monotone cocoercive operator. This latter framework has a number of interesting special cases, including variational inequalities and convex minimization problems, while stochastic approaches are practically relevant to account for perturbations in the data. The algorithm we propose is a stochastic extension of the classical deterministic forward-backward method, and is obtained considering the composition of the resolvent of the maximal monotone operator with a forward step based on a stochastic estimate of the single-valued operator. Our study provides a non-asymptotic error analysis in expectation for the strongly monotone case, as well as almost sure convergence under weaker assumptions. The approach we consider allows to avoid averaging, a feature critical when considering methods based on sparsity, and, for minimization problems, it allows to obtain convergence rates matching those obtained by stochastic extensions of so called accelerated methods. Stochastic quasi Fejer's sequences are a key technical tool to prove almost sure convergence.

20 pages

References in corpus (2)

• Stochastic Quasi-Fejér Block-Coordinate Fixed Point Iterations with Random Sweeping
• Variable Metric Quasi-Fejér Monotonicity

Cited by in corpus (10)

• A Variational Inequality Perspective on Generative Adversarial Networks
• A Survey of Stochastic Simulation and Optimization Methods in Signal Processing
• Stochastic Approximations and Perturbations in Forward-Backward Splitting for Monotone Operators
• Stochastic Variance Reduction Methods for Saddle-Point Problems
• A Stochastic Proximal Point Algorithm for Saddle-Point Problems
• Tight last-iterate convergence rates for no-regret learning in multi-player games
• Stochastic inertial primal-dual algorithms
• A stochastic inertial forward-backward splitting algorithm for multivariate monotone inclusions
• Stochastic Gradient Descent-Ascent and Consensus Optimization for Smooth Games: Convergence Analysis under Expected Co-coercivity
• Almost sure convergence of the forward-backward-forward splitting algorithm