8 citations · 26 across the 11 of their papers we have counts for
11 papers
NRGPT: An Energy-based Alternative for GPT
Nima Dehmamy, Benjamin Hoover, Bishwajit Saha +3
Generative Pre-trained Transformer (GPT) architectures are the most popular design for language modeling. Energy-based modeling is a different paradigm that views inference as a dy…
InputDSA: Demixing then Comparing Recurrent and Externally Driven Dynamics
Ann Huang, Mitchell Ostrow, Satpreet H. Singh +3
In control problems and basic scientific modeling, it is important to compare observations with dynamical simulations. For example, comparing two neural systems can shed light on t…
Characterizing control between interacting subsystems with deep Jacobian estimation
Adam J. Eisen, Mitchell Ostrow, Sarthak Chandra +3
Biological function arises through the dynamical interactions of multiple subsystems, including those between brain areas, within gene regulatory networks, and more. A common appro…
Neuron-Astrocyte Associative Memory
Leo Kozachkov, Jean-Jacques Slotine, Dmitry Krotov
Astrocytes, the most abundant type of glial cell, play a fundamental role in memory. Despite most hippocampal synapses being contacted by an astrocyte, there are no current theorie…
Contraction Properties of the Global Workspace Primitive
Michaela Ennis, Leo Kozachkov, Jean-Jacques Slotine
To push forward the important emerging research field surrounding multi-area recurrent neural networks (RNNs), we expand theoretically and empirically on the provably stable RNNs o…
Beyond Geometry: Comparing the Temporal Structure of Computation in Neural Circuits with Dynamical Similarity Analysis
Mitchell Ostrow, Adam Eisen, Leo Kozachkov +1
How can we tell whether two neural networks utilize the same internal processes for a particular computation? This question is pertinent for multiple subfields of neuroscience and…
Matrix Measure Flows: A Novel Approach to Stable Plasticity in Neural Networks
Leo Kozachkov, Jean-Jacques Slotine
This letter introduces the notion of a matrix measure flow as a tool for analyzing the stability of neural networks with time-varying weights. Given a matrix flow -- for example, o…
Generalization in Supervised Learning Through Riemannian Contraction
Leo Kozachkov, Patrick M. Wensing, Jean-Jacques Slotine
We prove that Riemannian contraction in a supervised learning setting implies generalization. Specifically, we show that if an optimizer is contracting in some Riemannian metric wi…
RNNs of RNNs: Recursive Construction of Stable Assemblies of Recurrent Neural Networks
Leo Kozachkov, Michaela Ennis, Jean-Jacques Slotine
Recurrent neural networks (RNNs) are widely used throughout neuroscience as models of local neural activity. Many properties of single RNNs are well characterized theoretically, bu…
Sequence learning in Associative Neuronal-Astrocytic Network
Leo Kozachkov, Konstantinos P. Michmizos
The neuronal paradigm of studying the brain has left us with limitations in both our understanding of how neurons process information to achieve biological intelligence and how suc…
The Causal Role of Astrocytes in Slow-Wave Rhythmogenesis: A Computational Modelling Study
Leo Kozachkov, Konstantinos P. Michmizos
Finding the origin of slow and infra-slow oscillations could reveal or explain brain mechanisms in health and disease. Here, we present a biophysically constrained computational mo…