Flexible Parallel Algorithms for Big Data Optimization

Abstract

We propose a decomposition framework for the parallel optimization of the sum of a differentiable function and a (block) separable nonsmooth, convex one. The latter term is typically used to enforce structure in the solution as, for example, in Lasso problems. Our framework is very flexible and includes both fully parallel Jacobi schemes and Gauss-Seidel (Southwell-type) ones, as well as virtually all possibilities in between (e.g., gradient- or Newton-type methods) with only a subset of variables updated at each iteration. Our theoretical convergence results improve on existing ones, and numerical results show that the new method compares favorably to existing algorithms.

submitted to IEEE ICASSP 2014

References in corpus (1)

• Parallel Coordinate Descent for L1-Regularized Loss Minimization

Cited by in corpus (6)

• A Block Successive Upper Bound Minimization Method of Multipliers for Linearly Constrained Convex Optimization
• A distributed block coordinate descent method for training $l_1$ regularized linear classifiers
• Asynchronous Stochastic Coordinate Descent: Parallelism and Convergence Properties
• Computing B-Stationary Points of Nonsmooth DC Programs
• A dominance maximization approach to portfolio selection
• An Adaptive Partial Linearization Method for Optimization Problems on Product Sets