Block-Cyclic Stochastic Coordinate Descent for Deep Neural Networks

Kensuke Nakamura, Stefano Soatto, Byung-Woo Hong

We present a stochastic first-order optimization algorithm, named BCSC, that adds a cyclic constraint to stochastic block-coordinate descent. It uses different subsets of the data…