A Primer on Coordinate Descent Algorithms

Abstract

This monograph presents a class of algorithms called coordinate descent algorithms for mathematicians, statisticians, and engineers outside the field of optimization. This particular class of algorithms has recently gained popularity due to their effectiveness in solving large-scale optimization problems in machine learning, compressed sensing, image processing, and computational statistics. Coordinate descent algorithms solve optimization problems by successively minimizing along each coordinate or coordinate hyperplane, which is ideal for parallelized and distributed computing. Avoiding detailed technicalities and proofs, this monograph gives relevant theory and examples for practitioners to effectively apply coordinate descent to modern problems in data science and engineering.

References in corpus (12)

• SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives
• Parallel Coordinate Descent for L1-Regularized Loss Minimization
• Even Faster Accelerated Coordinate Descent Using Non-Uniform Sampling
• Stochastic Primal-Dual Coordinate Method for Regularized Empirical Risk Minimization
• Coordinate Descent Converges Faster with the Gauss-Southwell Rule Than Random Selection
• Coordinate Friendly Structures, Algorithms and Applications
• Distributed Block Coordinate Descent for Minimizing Partially Separable Functions
• Smooth minimization of nonsmooth functions with parallel coordinate descent methods
• Iteration Complexity Analysis of Block Coordinate Descent Methods
• Sparse Bilinear Logistic Regression
• Randomized Primal-Dual Proximal Block Coordinate Updates
• TMAC: A Toolbox of Modern Async-Parallel, Coordinate, Splitting, and Stochastic Methods

Cited by in corpus (16)

• On computing the distance to stability for matrices using linear dissipative Hamiltonian systems
• Towards Overfitting Avoidance: Tuning-free Tensor-aided Multi-user Channel Estimation for 3D Massive MIMO Communications
• Optimal Orbital Selection for Full Configuration Interaction (OptOrbFCI): Pursuing the Basis Set Limit under a Budget
• From safe screening rules to working sets for faster Lasso-type solvers
• Cooperative Coevolution for Non-Separable Large-Scale Black-Box Optimization: Convergence Analyses and Distributed Accelerations
• Model identification and local linear convergence of coordinate descent
• A Survey on Large-scale Machine Learning
• Improved Search Strategies with Application to Estimating Facial Blendshape Parameters
• On the global convergence of randomized coordinate gradient descent for non-convex optimization
• High-Dimensional Private Empirical Risk Minimization by Greedy Coordinate Descent
• Anderson acceleration of coordinate descent
• Cyclic Coordinate Update Algorithms for Fixed-Point Problems: Analysis and Applications
• Asymptotic Statistical Analysis of Sparse Group LASSO via Approximate Message Passing Algorithm
• A generic coordinate descent solver for nonsmooth convex optimization
• Interpolation-based coordinate descent method for parameterized quantum circuits
• Harmonic Center of a Closed Convex Polytope: Definition, Calculation and Some Properties