Showing 2025Show all
2 papers · 1 filter
cs.LG2025
Do Neural Networks Need Gradient Descent to Generalize? A Theoretical Study
Yotam Alexander, Yonatan Slutzky, Yuval Ran-Milo +1
Conventional wisdom attributes the mysterious generalization abilities of overparameterized neural networks to gradient descent (and its variants). The recent volume hypothesis cha…
cs.LG2025
Revisiting Glorot Initialization for Long-Range Linear Recurrences
Noga Bar, Mariia Seleznova, Yotam Alexander +2
Proper initialization is critical for Recurrent Neural Networks (RNNs), particularly in long-range reasoning tasks, where repeated application of the same weight matrix can cause v…