355 citations · 956 across the 20 of their papers we have counts for
5 papers · 1 filter
NeurIPS 2020 Competition: Predicting Generalization in Deep Learning
Yiding Jiang, Pierre Foret, Scott Yak +7
Understanding generalization in deep learning is arguably one of the most important questions in deep learning. Deep learning has been successfully adopted to a large number of pro…
Data Augmentation via Structured Adversarial Perturbations
Calvin Luo, Hossein Mobahi, Samy Bengio
Data augmentation is a major component of many machine learning methods with state-of-the-art performance. Common augmentation strategies work by drawing random samples from a spac…
A Unifying View on Implicit Bias in Training Linear Neural Networks
Chulhee Yun, Shankar Krishnan, Hossein Mobahi
We study the implicit bias of gradient flow (i.e., gradient descent with infinitesimal step size) on linear neural network training. We propose a tensor formulation of neural netwo…
Sharpness-Aware Minimization for Efficiently Improving Generalization
Pierre Foret, Ariel Kleiner, Hossein Mobahi +1
In today's heavily overparameterized models, the value of the training loss provides few guarantees on model generalization ability. Indeed, optimizing only the training loss value…
Self-Distillation Amplifies Regularization in Hilbert Space
Hossein Mobahi, Mehrdad Farajtabar, Peter L. Bartlett
Knowledge distillation introduced in the deep learning context is a method to transfer knowledge from one architecture to another. In particular, when the architectures are identic…