3 papers
cs.LG2020
Stochastic batch size for adaptive regularization in deep network optimization
Kensuke Nakamura, Stefano Soatto, Byung-Woo Hong
We propose a first-order stochastic optimization algorithm incorporating adaptive regularization applicable to machine learning problems in deep learning framework. The adaptive re…
cs.LG2019
Adaptive Weight Decay for Deep Neural Networks
Kensuke Nakamura, Byung-Woo Hong
Regularization in the optimization of deep neural networks is often critical to avoid undesirable over-fitting leading to better generalization of model. One of the most popular re…
cs.CV2017
Block-Cyclic Stochastic Coordinate Descent for Deep Neural Networks
Kensuke Nakamura, Stefano Soatto, Byung-Woo Hong
We present a stochastic first-order optimization algorithm, named BCSC, that adds a cyclic constraint to stochastic block-coordinate descent. It uses different subsets of the data…