22 citations · 45 across the 13 of their papers we have counts for
Showing 2022Show all
3 papers · 1 filter
cs.LG2022★ 1 cited
Understanding Edge-of-Stability Training Dynamics with a Minimalist Example
Xingyu Zhu, Zixuan Wang, Xiang Wang +2
Recently, researchers observed that gradient descent for deep neural networks operates in an ``edge-of-stability'' (EoS) regime: the sharpness (maximum eigenvalue of the Hessian) i…
stat.ML2022
Plateau in Monotonic Linear Interpolation -- A "Biased" View of Loss Landscape for Deep Networks
Xiang Wang, Annie N. Wang, Mo Zhou +1
Monotonic linear interpolation (MLI) - on the line connecting a random initialization with the minimizer it converges to, the loss and accuracy are monotonic - is a phenomenon that…
cs.CL2022
Understanding The Robustness of Self-supervised Learning Through Topic Modeling
Zeping Luo, Shiyou Wu, Cindy Weng +2
Self-supervised learning has significantly improved the performance of many NLP tasks. However, how can self-supervised learning discover useful representations, and why is it bett…