2.1k citations · 2.8k across the 13 of their papers we have counts for
6 papers · 1 filter
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time
Mitchell Wortsman, Gabriel Ilharco, Samir Yitzhak Gadre +8
The conventional recipe for maximizing model accuracy is to (1) train multiple models with various hyperparameters and (2) pick the individual model which performs best on a held-o…
No One Representation to Rule Them All: Overlapping Features of Training Methods
Raphael Gontijo-Lopes, Yann Dauphin, Ekin D. Cubuk
Despite being able to capture a range of features of the data, high accuracy models trained with supervision tend to make similar predictions. This seemingly implies that high-perf…
Spectral Bias in Practice: The Role of Function Frequency in Generalization
Sara Fridovich-Keil, Raphael Gontijo-Lopes, Rebecca Roelofs
Despite their ability to represent highly expressive functions, deep learning models seem to find simple solutions that generalize surprisingly well. Spectral bias -- the tendency…
Improving Robustness Without Sacrificing Accuracy with Patch Gaussian Augmentation
Raphael Gontijo Lopes, Dong Yin, Ben Poole +2
Deploying machine learning systems in the real world requires both high accuracy on clean data and robustness to naturally occurring corruptions. While architectural advances have…
A Fourier Perspective on Model Robustness in Computer Vision
Dong Yin, Raphael Gontijo Lopes, Jonathon Shlens +2
Achieving robustness to distributional shift is a longstanding and challenging goal of computer vision. Data augmentation is a commonly used approach for improving robustness, howe…
Data-Free Knowledge Distillation for Deep Neural Networks
Raphael Gontijo Lopes, Stefano Fenu, Thad Starner
Recent advances in model compression have provided procedures for compressing large neural networks to a fraction of their original size while retaining most if not all of their ac…