Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
Implicit Bias of SGD in Multivariate ReLU Networks: Effective Width Collapse
Shuang Liang, Tom Jacobs, Guido Montúfar
We study the implicit bias of noisy stochastic gradient descent in training wide two-layer ReLU networks for multivariate regression. In a mean-field regime, the training dynamics…
cs.LG2025
Depth-induced NTK: Bridging Over-parameterized Neural Networks and Deep Neural Kernels
Yong-Ming Tian, Shuang Liang, Shao-Qun Zhang +1
While deep learning has achieved remarkable success across a wide range of applications, its theoretical understanding of representation learning remains limited. Deep neural kerne…
cs.LG2025
Gradient Descent with Large Step Sizes: Chaos and Fractal Convergence Region
Shuang Liang, Guido Montúfar
We examine gradient descent in matrix factorization and show that under large step sizes the parameter space develops a fractal structure. We derive the exact critical step size fo…