353 citations · 1.2k across the 32 of their papers we have counts for
5 papers · 1 filter
Variational Bi-LSTMs
Samira Shabanian, Devansh Arpit, Adam Trischler +1
Recurrent neural networks like long short-term memory (LSTM) are important architectures for sequential prediction tasks. LSTMs (and RNNs in general) model sequences along the forw…
Three Factors Influencing Minima in SGD
Stanisław Jastrzębski, Zachary Kenton, Devansh Arpit +4
We investigate the dynamical and convergent properties of stochastic gradient descent (SGD) applied to Deep Neural Networks (DNNs). Characterizing the relation between learning rat…
Fraternal Dropout
Konrad Zolna, Devansh Arpit, Dendi Suhubdy +1
Recurrent neural networks (RNNs) are important class of architectures among neural networks useful for language modeling and sequential prediction. However, optimizing RNNs is know…
Residual Connections Encourage Iterative Inference
Stanisław Jastrzębski, Devansh Arpit, Nicolas Ballas +3
Residual networks (Resnets) have become a prominent architecture in deep learning. However, a comprehensive understanding of Resnets is still a topic of ongoing research. A recent…
A Closer Look at Memorization in Deep Networks
Devansh Arpit, Stanisław Jastrzębski, Nicolas Ballas +8
We examine the role of memorization in deep learning, drawing connections to capacity, generalization, and adversarial robustness. While deep networks are capable of memorizing noi…