Adaptive Regularization via Residual Smoothing in Deep Learning Optimization

Junghee Cho, Junseok Kwon, Byung-Woo Hong

We present an adaptive regularization algorithm that can be effectively applied to the optimization problem in deep learning framework. Our regularization algorithm aims to take in…