565 citations · 612 across the 9 of their papers we have counts for
9 papers
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao +448
Language models demonstrate both quantitative improvement and new qualitative capabilities with increasing scale. Despite their potentially transformative impact, these new capabil…
Deep Networks Provably Classify Data on Curves
Tingran Wang, Sam Buchanan, Dar Gilboa +1
Data with low-dimensional nonlinear structure are ubiquitous in engineering and scientific problems. We study a model problem with such structure -- a binary classification task th…
Deep Networks and the Multiple Manifold Problem
Sam Buchanan, Dar Gilboa, John Wright
We study the multiple manifold problem, a binary classification task modeled on applications in machine vision, in which a deep fully-connected neural network is trained to separat…
Beyond Signal Propagation: Is Feature Diversity Necessary in Deep Neural Network Initialization?
Yaniv Blumenfeld, Dar Gilboa, Daniel Soudry
Deep neural networks are typically initialized with random weights, with variances chosen to facilitate signal propagation and stable gradients. It is also believed that diversity…
Is Feature Diversity Necessary in Neural Network Initialization?
Yaniv Blumenfeld, Dar Gilboa, Daniel Soudry
Standard practice in training neural networks involves initializing the weights in an independent fashion. The results of recent work suggest that feature "diversity" at initializa…
Wider Networks Learn Better Features
Dar Gilboa, Guy Gur-Ari
Transferability of learned features between tasks can massively reduce the cost of training a neural network on a novel task. We investigate the effect of network width on learned…
A Mean Field Theory of Quantized Deep Networks: The Quantization-Depth Trade-Off
Yaniv Blumenfeld, Dar Gilboa, Daniel Soudry
Reducing the precision of weights and activation functions in neural network training, with minimal impact on performance, is essential for the deployment of these models in resour…
Dynamical Isometry and a Mean Field Theory of LSTMs and GRUs
Dar Gilboa, Bo Chang, Minmin Chen +4
Training recurrent neural networks (RNNs) on long sequence tasks is plagued with difficulties arising from the exponential explosion or vanishing of signals as they propagate forwa…
Efficient Dictionary Learning with Gradient Descent
Dar Gilboa, Sam Buchanan, John Wright
Randomly initialized first-order optimization algorithms are the method of choice for solving many high-dimensional nonconvex problems in machine learning, yet general theoretical…