A Random Block-Coordinate Douglas-Rachford Splitting Method with Low Computational Complexity for Binary Logistic Regression
arXiv:1712.09131 · doi:10.1007/s10589-019-00060-6
Abstract
In this paper, we propose a new optimization algorithm for sparse logistic regression based on a stochastic version of the Douglas-Rachford splitting method. Our algorithm sweeps the training set by randomly selecting a mini-batch of data at each iteration, and it allows us to update the variables in a block coordinate manner. Our approach leverages the proximity operator of the logistic loss, which is expressed with the generalized Lambert W function. Experiments carried out on standard datasets demonstrate the efficiency of our approach w.r.t. stochastic gradient-like methods.
References in corpus (11)
- From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification
- Sparse Online Learning via Truncated Gradient
- A Survey of Stochastic Simulation and Optimization Methods in Signal Processing
- Block-Coordinate Frank-Wolfe Optimization for Structural SVMs
- Optimization with First-Order Surrogate Functions
- Non-convex Regularizations for Feature Selection in Ranking With Sparse SVM
- Stochastic Majorization-Minimization Algorithms for Large-Scale Optimization
- Nonparametric sparsity and regularization
- On perturbed proximal gradient algorithms
- Stochastic Approximations and Perturbations in Forward-Backward Splitting for Monotone Operators
- Projective Splitting with Forward Steps: Asynchronous and Block-Iterative Operator Splitting