776 citations · 2.2k across the 24 of their papers we have counts for
16 papers · 1 filter
Proximity Preserving Binary Code using Signed Graph-Cut
Inbal Lav, Shai Avidan, Yoram Singer +1
We introduce a binary embedding framework, called Proximity Preserving Code (PPC), which learns similarity and dissimilarity between data points to create a compact and affinity-pr…
Scalable Second Order Optimization for Deep Learning
Rohan Anil, Vineet Gupta, Tomer Koren +2
Optimization in machine learning, both theoretical and applied, is presently dominated by first-order gradient methods such as stochastic gradient descent. Second-order optimizatio…
Convolutional Bipartite Attractor Networks
Michael Iuzzolino, Yoram Singer, Michael C. Mozer
In human perception and cognition, a fundamental operation that brains perform is interpretation: constructing coherent neural states from noisy, incomplete, and intrinsically ambi…
Exponentiated Gradient Meets Gradient Descent
Udaya Ghai, Elad Hazan, Yoram Singer
The (stochastic) gradient descent and the multiplicative update method are probably the most popular algorithms in machine learning. We introduce and study a new regularization whi…
Memory-Efficient Adaptive Optimization
Rohan Anil, Vineet Gupta, Tomer Koren +1
Adaptive gradient-based optimizers such as Adagrad and Adam are crucial for achieving state-of-the-art performance in machine translation and language modeling. However, these meth…
Shampoo: Preconditioned Stochastic Tensor Optimization
Vineet Gupta, Tomer Koren, Yoram Singer
Preconditioned gradient methods are among the most general and powerful tools in optimization. However, preconditioning requires storing and manipulating prohibitively large matric…