24 citations · 33 across the 4 of their papers we have counts for
4 papers · 1 filter
Beyond Signal Propagation: Is Feature Diversity Necessary in Deep Neural Network Initialization?
Yaniv Blumenfeld, Dar Gilboa, Daniel Soudry
Deep neural networks are typically initialized with random weights, with variances chosen to facilitate signal propagation and stable gradients. It is also believed that diversity…
Is Feature Diversity Necessary in Neural Network Initialization?
Yaniv Blumenfeld, Dar Gilboa, Daniel Soudry
Standard practice in training neural networks involves initializing the weights in an independent fashion. The results of recent work suggest that feature "diversity" at initializa…
Wider Networks Learn Better Features
Dar Gilboa, Guy Gur-Ari
Transferability of learned features between tasks can massively reduce the cost of training a neural network on a novel task. We investigate the effect of network width on learned…
Dynamical Isometry and a Mean Field Theory of LSTMs and GRUs
Dar Gilboa, Bo Chang, Minmin Chen +4
Training recurrent neural networks (RNNs) on long sequence tasks is plagued with difficulties arising from the exponential explosion or vanishing of signals as they propagate forwa…