On ADMM in Deep Learning: Convergence and Saturation-Avoidance

Jinshan Zeng, Shao-Bo Lin, Yuan Yao +1

In this paper, we develop an alternating direction method of multipliers (ADMM) for deep neural networks training with sigmoid-type activation functions (called \textit{sigmoid-ADM…