6 citations · 29 across the 17 of their papers we have counts for
8 papers · 1 filter
Traversing the solution space of neural networks with Hessian Null Space Continuation
Ann Huang, Mitchell Ostrow, Zhouyang Lu +3
On a single task, deep networks can learn many solutions, depending on their optimizer, training data, architecture, and hyperparameters. Many of these solutions are mode-connected…
Shortcut Solutions Learned by Transformers Impair Continual Compositional Reasoning
William T. Redman, Erik C. Johnson, Brian Robinson
Identifying and exploiting common features across domains is at the heart of the human ability to make analogies, and is believed to be crucial for the ability to continually learn…
On How Iterative Magnitude Pruning Discovers Local Receptive Fields in Fully Connected Neural Networks
William T. Redman, Zhangyang Wang, Alessandro Ingrosso +1
Since its use in the Lottery Ticket Hypothesis, iterative magnitude pruning (IMP) has become a popular method for extracting sparse subnetworks that can be trained to high performa…
Learning Linear Embeddings for Non-Linear Network Dynamics with Koopman Message Passing
King Fai Yeh, Paris Flood, William Redman +1
Recently, Koopman operator theory has become a powerful tool for developing linear representations of non-linear dynamical systems. However, existing data-driven applications of Ko…
Identifying Equivalent Training Dynamics
William T. Redman, Juan M. Bello-Rivas, Maria Fonoberova +3
Study of the nonlinear evolution deep neural network (DNN) parameters undergo during training has uncovered regimes of distinct dynamical behavior. While a detailed understanding o…
An Operator Theoretic View on Pruning Deep Neural Networks
William T. Redman, Maria Fonoberova, Ryan Mohr +2
The discovery of sparse subnetworks that are able to perform as well as full models has found broad applied and theoretical interest. While many pruning methods have been developed…
Universality of Winning Tickets: A Renormalization Group Perspective
William T. Redman, Tianlong Chen, Zhangyang Wang +1
Foundational work on the Lottery Ticket Hypothesis has suggested an exciting corollary: winning tickets found in the context of one task can be transferred to similar tasks, possib…
Local error quantification for Neural Network Differential Equation solvers
Akshunna S. Dogra, William T Redman
Neural networks have been identified as powerful tools for the study of complex systems. A noteworthy example is the neural network differential equation (NN DE) solver, which can…