78 citations · 165 across the 10 of their papers we have counts for
5 papers · 1 filter
h-detach: Modifying the LSTM Gradient Towards Better Optimization
Devansh Arpit, Bhargav Kanuparthi, Giancarlo Kerg +3
Recurrent neural networks are known for their notorious exploding and vanishing gradient problem (EVGP). This problem becomes more evident in tasks where the information needed to…
Manifold Mixup: Better Representations by Interpolating Hidden States
Vikas Verma, Alex Lamb, Christopher Beckham +5
Deep neural networks excel at learning the training data, but often provide incorrect and confident predictions when evaluated on slightly different test examples. This includes di…
Fortified Networks: Improving the Robustness of Deep Networks by Modeling the Manifold of Hidden Representations
Alex Lamb, Jonathan Binas, Anirudh Goyal +4
Deep networks have achieved impressive results across a variety of important tasks. However a known weakness is a failure to perform well when evaluated on data which differ from t…
Improving Gibbs Sampler Scan Quality with DoGS
Ioannis Mitliagkas, Lester Mackey
The pairwise influence matrix of Dobrushin has long been used as an analytical tool to bound the rate of convergence of Gibbs sampling. In this work, we use Dobrushin influence as…
Parallel SGD: When does averaging help?
Jian Zhang, Christopher De Sa, Ioannis Mitliagkas +1
Consider a number of workers running SGD independently on the same pool of data and averaging the models every once in a while -- a common but not well understood practice. We stud…