17 citations · 38 across the 7 of their papers we have counts for
7 papers
The Butterfly Effect: Neural Network Training Trajectories Are Highly Sensitive to Initial Conditions
Devin Kwok, Gül Sena Altıntaş, Colin Raffel +1
Neural network training is inherently sensitive to initialization and the randomness induced by stochastic gradient descent. However, it is unclear to what extent such effects lead…
Simultaneous linear connectivity of neural networks modulo permutation
Ekansh Sharma, Devin Kwok, Tom Denton +3
Neural networks typically exhibit permutation symmetries which contribute to the non-convexity of the networks' loss landscapes, since linearly interpolating between two permuted v…
Dataset Difficulty and the Role of Inductive Bias
Devin Kwok, Nikhil Anand, Jonathan Frankle +2
Motivated by the goals of dataset pruning and defect identification, a growing body of methods have been developed to score individual examples within a dataset. These methods, whi…
Neural Networks as Paths through the Space of Representations
Richard D. Lange, Devin Kwok, Jordan Matelsky +3
Deep neural networks implement a sequence of layer-by-layer operations that are each relatively easy to understand, but the resulting overall computation is generally difficult to…
Clustering units in neural networks: upstream vs downstream information
Richard D. Lange, David S. Rolnick, Konrad P. Kording
It has been hypothesized that some form of "modular" structure in artificial neural networks should be useful for learning, compositionality, and generalization. However, defining…
Reverse-Engineering Deep ReLU Networks
David Rolnick, Konrad P. Kording
It has been widely assumed that a neural network cannot be recovered from its outputs, as the network depends on its parameters in a highly nonlinear way. Here, we prove that in fa…