776 citations · 2.2k across the 24 of their papers we have counts for
5 papers · 1 filter
Convolutional Bipartite Attractor Networks
Michael Iuzzolino, Yoram Singer, Michael C. Mozer
In human perception and cognition, a fundamental operation that brains perform is interpretation: constructing coherent neural states from noisy, incomplete, and intrinsically ambi…
Exponentiated Gradient Meets Gradient Descent
Udaya Ghai, Elad Hazan, Yoram Singer
The (stochastic) gradient descent and the multiplicative update method are probably the most popular algorithms in machine learning. We introduce and study a new regularization whi…
Identity Crisis: Memorization and Generalization under Extreme Overparameterization
Chiyuan Zhang, Samy Bengio, Moritz Hardt +2
We study the interplay between memorization and generalization of overparameterized networks in the extreme case of a single training example and an identity-mapping task. We exami…
Are All Layers Created Equal?
Chiyuan Zhang, Samy Bengio, Yoram Singer
Understanding deep neural networks is a major research objective with notable experimental and theoretical attention in recent years. The practical success of excessively large net…
Memory-Efficient Adaptive Optimization
Rohan Anil, Vineet Gupta, Tomer Koren +1
Adaptive gradient-based optimizers such as Adagrad and Adam are crucial for achieving state-of-the-art performance in machine translation and language modeling. However, these meth…