2k citations · 4.8k across the 35 of their papers we have counts for
3 papers · 1 filter
Three Factors Influencing Minima in SGD
Stanisław Jastrzębski, Zachary Kenton, Devansh Arpit +4
We investigate the dynamical and convergent properties of stochastic gradient descent (SGD) applied to Deep Neural Networks (DNNs). Characterizing the relation between learning rat…
Residual Connections Encourage Iterative Inference
Stanisław Jastrzębski, Devansh Arpit, Nicolas Ballas +3
Residual networks (Resnets) have become a prominent architecture in deep learning. However, a comprehensive understanding of Resnets is still a topic of ongoing research. A recent…
A Closer Look at Memorization in Deep Networks
Devansh Arpit, Stanisław Jastrzębski, Nicolas Ballas +8
We examine the role of memorization in deep learning, drawing connections to capacity, generalization, and adversarial robustness. While deep networks are capable of memorizing noi…