108 citations · 265 across the 6 of their papers we have counts for
7 papers
Asymptotics of Wide Convolutional Neural Networks
Anders Andreassen, Ethan Dyer
Wide neural networks have proven to be a rich class of architectures for both theory and practice. Motivated by the observation that finite width convolutional networks appear to o…
Anatomy of Catastrophic Forgetting: Hidden Representations and Task Semantics
Vinay V. Ramasesh, Ethan Dyer, Maithra Raghu
A central challenge in developing versatile machine learning systems is catastrophic forgetting: a model trained on tasks in sequence will suffer significant performance drops on e…
The large learning rate phase of deep learning: the catapult mechanism
Aitor Lewkowycz, Yasaman Bahri, Ethan Dyer +2
The choice of initial learning rate can have a profound effect on the performance of deep networks. We present a class of neural networks with solvable training dynamics, and confi…
Affinity and Diversity: Quantifying Mechanisms of Data Augmentation
Raphael Gontijo-Lopes, Sylvia J. Smullin, Ekin D. Cubuk +1
Though data augmentation has become a standard component of deep neural network training, the underlying mechanism behind the effectiveness of these techniques remains poorly under…
Asymptotics of Wide Networks from Feynman Diagrams
Ethan Dyer, Guy Gur-Ari
Understanding the asymptotic behavior of wide networks is of considerable interest. In this work, we present a general method for analyzing this large width behavior. The method is…
The Most Irrational Rational Theories
Nathan Benjamin, Ethan Dyer, A. Liam Fitzpatrick +1
We propose a two-parameter family of modular invariant partition functions of two-dimensional conformal field theories (CFTs) holographically dual to pure three-dimensional gravity…