183 citations · 387 across the 16 of their papers we have counts for
15 papers · 1 filter
Pruning's Effect on Generalization Through the Lens of Training and Regularization
Tian Jin, Michael Carbin, Daniel M. Roy +2
Practitioners frequently observe that pruning improves model generalization. A long-standing hypothesis based on bias-variance trade-off attributes this generalization improvement…
NUQSGD: Provably Communication-efficient Data-parallel SGD via Nonuniform Quantization
Ali Ramezani-Kebrya, Fartash Faghri, Ilya Markov +3
As the size and complexity of models and datasets grow, so does the need for communication-efficient variants of stochastic gradient descent that can be deployed to perform paralle…
NeurIPS 2020 Competition: Predicting Generalization in Deep Learning
Yiding Jiang, Pierre Foret, Scott Yak +7
Understanding generalization in deep learning is arguably one of the most important questions in deep learning. Deep learning has been successfully adopted to a large number of pro…
On the Information Complexity of Proper Learners for VC Classes in the Realizable Case
Mahdi Haghifam, Gintare Karolina Dziugaite, Shay Moran +1
We provide a negative resolution to a conjecture of Steinke and Zakynthinou (2020a), by showing that their bound on the conditional mutual information (CMI) of proper learners of V…
Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the Neural Tangent Kernel
Stanislav Fort, Gintare Karolina Dziugaite, Mansheej Paul +3
In suitably initialized wide networks, small learning rates transform deep neural networks (DNNs) into neural tangent kernel (NTK) machines, whose training dynamics is well-approxi…
Enforcing Interpretability and its Statistical Impacts: Trade-offs between Accuracy and Interpretability
Gintare Karolina Dziugaite, Shai Ben-David, Daniel M. Roy
To date, there has been no formal study of the statistical cost of interpretability in machine learning. As such, the discourse around potential trade-offs is often informal and mi…