6 citations · 29 across the 17 of their papers we have counts for
18 papers
Traversing the solution space of neural networks with Hessian Null Space Continuation
Ann Huang, Mitchell Ostrow, Zhouyang Lu +3
On a single task, deep networks can learn many solutions, depending on their optimizer, training data, architecture, and hyperparameters. Many of these solutions are mode-connected…
Shortcut Solutions Learned by Transformers Impair Continual Compositional Reasoning
William T. Redman, Erik C. Johnson, Brian Robinson
Identifying and exploiting common features across domains is at the heart of the human ability to make analogies, and is believed to be crucial for the ability to continually learn…
Interpreting Reinforcement Learning Model Behavior via Koopman with Control
William T. Redman
Reinforcement learning (RL) models have shown the capability of learning complex behaviors, but quantitatively assessing those behaviors - which is critical for safety assurance an…
Trojans in Artificial Intelligence (TrojAI) Final Report
Kristopher W. Reese, Taylor Kulp-McDowall, Michael Majurski +68
The Intelligence Advanced Research Projects Activity (IARPA) launched the TrojAI program to confront an emerging vulnerability in modern artificial intelligence: the threat of AI T…
Distance by de-correlation: Computing distance with heterogeneous grid cells
Pritipriya Dasbehera, Akshunna S. Dogra, William T. Redman
Encoding the distance between locations in space is essential for accurate navigation. Grid cells, a functional class of neurons in medial entorhinal cortex, are believed to suppor…
On How Iterative Magnitude Pruning Discovers Local Receptive Fields in Fully Connected Neural Networks
William T. Redman, Zhangyang Wang, Alessandro Ingrosso +1
Since its use in the Lottery Ticket Hypothesis, iterative magnitude pruning (IMP) has become a popular method for extracting sparse subnetworks that can be trained to high performa…
Koopman Learning with Episodic Memory
William T. Redman, Dean Huang, Maria Fonoberova +1
Koopman operator theory has found significant success in learning models of complex, real-world dynamical systems, enabling prediction and control. The greater interpretability and…
Learning Linear Embeddings for Non-Linear Network Dynamics with Koopman Message Passing
King Fai Yeh, Paris Flood, William Redman +1
Recently, Koopman operator theory has become a powerful tool for developing linear representations of non-linear dynamical systems. However, existing data-driven applications of Ko…
Identifying Equivalent Training Dynamics
William T. Redman, Juan M. Bello-Rivas, Maria Fonoberova +3
Study of the nonlinear evolution deep neural network (DNN) parameters undergo during training has uncovered regimes of distinct dynamical behavior. While a detailed understanding o…
Algorithmic (Semi-)Conjugacy via Koopman Operator Theory
William T. Redman, Maria Fonoberova, Ryan Mohr +2
Iterative algorithms are of utmost importance in decision and control. With an ever growing number of algorithms being developed, distributed, and proprietarized, there is a simila…
An Operator Theoretic View on Pruning Deep Neural Networks
William T. Redman, Maria Fonoberova, Ryan Mohr +2
The discovery of sparse subnetworks that are able to perform as well as full models has found broad applied and theoretical interest. While many pruning methods have been developed…
Universality of Winning Tickets: A Renormalization Group Perspective
William T. Redman, Tianlong Chen, Zhangyang Wang +1
Foundational work on the Lottery Ticket Hypothesis has suggested an exciting corollary: winning tickets found in the context of one task can be transferred to similar tasks, possib…
On Koopman Mode Decomposition and Tensor Component Analysis
William T. Redman
Koopman mode decomposition and tensor component analysis (also known as CANDECOMP/PARAFAC or canonical polyadic decomposition) are two popular approaches of decomposing high dimens…
Local error quantification for Neural Network Differential Equation solvers
Akshunna S. Dogra, William T Redman
Neural networks have been identified as powerful tools for the study of complex systems. A noteworthy example is the neural network differential equation (NN DE) solver, which can…
Optimizing Neural Networks via Koopman Operator Theory
Akshunna S. Dogra, William T Redman
Koopman operator theory, a powerful framework for discovering the underlying dynamics of nonlinear dynamical systems, was recently shown to be intimately connected with neural netw…
Renormalization Group as a Koopman Operator
William T Redman
Koopman operator theory is shown to be directly related to the renormalization group. This observation allows us, with no assumption of translational invariance, to compute the cri…
An O(n) method of calculating Kendall correlations of spike trains
William T Redman
The ability to record from increasingly large numbers of neurons, and the increasing attention being paid to large scale neural network simulations, demands computationally fast al…
A General Approach to Coding in Early Olfactory and Visual Neural Populations
William T Redman
Recent experimental and theoretical work on neural populations belonging to two separate early sensory systems, olfaction and vision, has challenged the notion that the two operate…