4.7k citations · 10.6k across the 33 of their papers we have counts for
3 papers · 1 filter
Temperature check: theory and practice for training models with softmax-cross-entropy losses
Atish Agarwala, Jeffrey Pennington, Yann Dauphin +1
The softmax function combined with a cross-entropy loss is a principled approach to modeling probability distributions that has become ubiquitous in deep learning. The softmax func…
Gradient Flow in Sparse Neural Networks and How Lottery Tickets Win
Utku Evci, Yani A. Ioannou, Cem Keskin +1
Sparse Neural Networks (NNs) can match the generalization of dense NNs using a fraction of the compute/storage for inference, and also have the potential to enable efficient traini…
Robust and On-the-fly Dataset Denoising for Image Classification
Jiaming Song, Lunjia Hu, Michael Auli +2
Memorization in over-parameterized neural networks could severely hurt generalization in the presence of mislabeled examples. However, mislabeled examples are hard to avoid in extr…