A Random Block-Coordinate Douglas-Rachford Splitting Method with Low Computational Complexity for Binary Logistic Regression

Abstract

In this paper, we propose a new optimization algorithm for sparse logistic regression based on a stochastic version of the Douglas-Rachford splitting method. Our algorithm sweeps the training set by randomly selecting a mini-batch of data at each iteration, and it allows us to update the variables in a block coordinate manner. Our approach leverages the proximity operator of the logistic loss, which is expressed with the generalized Lambert W function. Experiments carried out on standard datasets demonstrate the efficiency of our approach w.r.t. stochastic gradient-like methods.

References in corpus (6)

• From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification
• Sparse Online Learning via Truncated Gradient
• Block-Coordinate Frank-Wolfe Optimization for Structural SVMs
• Non-convex Regularizations for Feature Selection in Ranking With Sparse SVM
• Nonparametric sparsity and regularization
• Projective Splitting with Forward Steps: Asynchronous and Block-Iterative Operator Splitting

Cited by in corpus (4)

• Fixed Point Strategies in Data Science
• A Warped Resolvent Algorithm to Construct Nash Equilibria
• Analysis and Numerical Solution of a Modular Convex Nash Equilibrium Problem
• Coordinate-Update Algorithms can Efficiently Detect Infeasible Optimization Problems