Understanding Optimization of Deep Learning via Jacobian Matrix and Lipschitz Constant

Xianbiao Qi, Jianan Wang, Lei Zhang

This article provides a comprehensive understanding of optimization in deep learning, with a primary focus on the challenges of gradient vanishing and gradient exploding, which nor…