A Stochastic Alternating Direction Method of Multipliers for Non-smooth and Non-convex Optimization

Abstract

Alternating direction method of multipliers (ADMM) is a popular first-order method owing to its simplicity and efficiency. However, similar to other proximal splitting methods, the performance of ADMM degrades significantly when the scale of the optimization problems to solve becomes large. In this paper, we consider combining ADMM with a class of stochastic gradient with variance reduction for solving large-scale non-convex and non-smooth optimization problems. Global convergence of the generated sequence is established under the extra additional assumption that the object function satisfies Kurdyka-Lojasiewicz (KL) property. Numerical experiments on graph-guided fused Lasso and computed tomography are presented to demonstrate the performance of the proposed methods.

References in corpus (15)

• SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives
• Global convergence of splitting methods for nonconvex composite optimization
• Mini-Batch Semi-Stochastic Gradient Descent in the Proximal Setting
• Calculus of the exponent of Kurdyka-Łojasiewicz inequality and its applications to linear convergence of first-order methods
• Online Alternating Direction Method
• SARAH: A Novel Method for Machine Learning Problems Using Stochastic Recursive Gradient
• Convergence of Bregman alternating direction method with multipliers for nonconvex composite problems
• The Asynchronous PALM Algorithm for Nonsmooth Nonconvex Problems
• Accelerated Variance Reduced Stochastic ADMM
• Faster Stochastic Alternating Direction Method of Multipliers for Nonconvex Optimization
• Linearized ADMM for Non-convex Non-smooth Optimization with Convergence Analysis
• Scalable Stochastic Alternating Direction Method of Multipliers
• An Accelerated Linearized Alternating Direction Method of Multipliers
• SPRING: A fast stochastic proximal alternating method for non-smooth non-convex optimization
• Mini-Batch Stochastic ADMMs for Nonconvex Nonsmooth Optimization

Cited by in corpus (1)

• Stochastic ADMM with batch size adaptation for nonconvex nonsmooth optimization