Implicit Bias of Gradient Descent for Non-Homogeneous Deep Networks

Yuhang Cai, Kangjie Zhou, Jingfeng Wu +3

We establish the asymptotic implicit bias of gradient descent (GD) for generic non-homogeneous deep networks under exponential loss. Specifically, we characterize three key propert…