Parallel Selective Algorithms for Big Data Optimization
arXiv:1402.5521 · doi:10.1109/TSP.2015.2399858
Abstract
We propose a decomposition framework for the parallel optimization of the sum of a differentiable (possibly nonconvex) function and a (block) separable nonsmooth, convex one. The latter term is usually employed to enforce structure in the solution, typically sparsity. Our framework is very flexible and includes both fully parallel Jacobi schemes and Gauss- Seidel (i.e., sequential) ones, as well as virtually all possibilities "in between" with only a subset of variables updated at each iteration. Our theoretical convergence results improve on existing ones, and numerical results on LASSO, logistic regression, and some nonconvex quadratic problems show that the new method consistently outperforms existing algorithms.
This work is an extended version of the conference paper that has been presented at IEEE ICASSP'14. The first and the second author contributed equally to the paper. This revised version contains new numerical results on non convex quadratic problems
References in corpus (2)
Cited by in corpus (34)
- Joint Optimization of Radio and Computational Resources for Multicell Mobile-Edge Computing
- Parallel and Distributed Methods for Nonconvex Optimization--Part II: Applications
- A Proximal Dual Consensus ADMM Method for Multi-Agent Constrained Optimization
- Hybrid Block Successive Approximation for One-Sided Non-Convex Min-Max Problems: Algorithms and Applications
- A Parallel Stochastic Approximation Method for Nonconvex Multi-Agent Optimization Problems
- A Unified Successive Pseudo-Convex Approximation Framework
- Inexact Block Coordinate Descent Algorithms for Nonsmooth Nonconvex Optimization
- Hybrid Random/Deterministic Parallel Algorithms for Nonconvex Big Data Optimization
- Successive Convex Approximation Algorithms for Sparse Signal Estimation with Nonconvex Regularizations
- A Unified Algorithmic Framework for Block-Structured Optimization Involving Big Data
- Distributed Adaptive Learning with Multiple Kernels in Diffusion Networks
- A Framework for Parallel and Distributed Training of Neural Networks
- Bi-Linear Modeling of Data Manifolds for Dynamic-MRI Recovery
- Distributed Nonconvex Multiagent Optimization Over Time-Varying Networks
- An Online Parallel and Distributed Algorithm for Recursive Estimation of Sparse Signals
- Asynchronous Parallel Algorithms for Nonconvex Big-Data Optimization. Part II: Complexity and Numerical Results
- On Nonconvex Decentralized Gradient Descent
- Asynchronous Decentralized Successive Convex Approximation
- DJAM: distributed Jacobi asynchronous method for learning personal models
- COKE: Communication-Censored Decentralized Kernel Learning
- Robust Block Coordinate Descent
- Randomized Block Proximal Methods for Distributed Stochastic Big-Data Optimization
- Energy-Efficient Data Collection and Wireless Power Transfer Using A MIMO Full-Duplex UAV
- NEXT: In-Network Nonconvex Optimization
- Piecewise linear regression and classification
- Distributed Stochastic Nonconvex Optimization and Learning based on Successive Convex Approximation
- A Class of Parallel Doubly Stochastic Algorithms for Large-Scale Learning
- Decentralized Dictionary Learning Over Time-Varying Digraphs
- PIANO: A Fast Parallel Iterative Algorithm for Multinomial and Sparse Multinomial Logistic Regression
- A Parallel Best-Response Algorithm with Exact Line Search for Nonconvex Sparsity-Regularized Rank Minimization
- A randomized primal distributed algorithm for partitioned and big-data non-convex optimization
- Distributed Partitioned Big-Data Optimization via Asynchronous Dual Decomposition
- Kernel Bi-Linear Modeling for Reconstructing Data on Manifolds: The Dynamic-MRI Case
- A core-set approach for distributed quadratic programming in big-data classification