440 citations · 1.6k across the 34 of their papers we have counts for
4 papers · 1 filter
A Modern Take on the Bias-Variance Tradeoff in Neural Networks
Brady Neal, Sarthak Mittal, Aristide Baratin +4
The bias-variance tradeoff tells us that as model complexity increases, bias falls and variances increases, leading to a U-shaped test error curve. However, recent empirical result…
On the Spectral Bias of Neural Networks
Nasim Rahaman, Aristide Baratin, Devansh Arpit +5
Neural networks are known to be a class of highly expressive functions able to fit even random input-output mappings with accuracy. In this work, we present properties of n…
A3T: Adversarially Augmented Adversarial Training
Akram Erraqabi, Aristide Baratin, Yoshua Bengio +1
Recent research showed that deep neural networks are highly sensitive to so-called adversarial perturbations, which are tiny perturbations of the input data purposely designed to f…
MINE: Mutual Information Neural Estimation
Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeswar +4
We argue that the estimation of mutual information between high dimensional continuous random variables can be achieved by gradient descent over neural networks. We present a Mutua…