12 citations · 13 across the 8 of their papers we have counts for
9 papers
Phase Transitions in Attention: A Bayesian Theory of Copy Head Emergence
Itay Lavie, Kirsten Fischer, Andrey Lekov +3
Attention is the key mechanism underlying in-context learning in transformers, and attention patterns have been observed empirically to emerge abruptly during training. We present…
A unified theory of feature learning in RNNs and DNNs
Jan P. Bauer, Kirsten Fischer, Moritz Helias +1
Recurrent and deep neural networks (RNNs/DNNs) are cornerstone architectures in machine learning. Remarkably, RNNs differ from DNNs only by weight sharing, as can be shown through…
Characterizing Neural Manifolds' Properties and Curvatures using Normalizing Flows
Peter Bouss, Sandra Nestler, Kirsten Fischer +3
Neuronal activity is found to lie on low-dimensional manifolds embedded within the high-dimensional neuron space. Variants of principal component analysis are frequently employed t…
From Kernels to Features: A Multi-Scale Adaptive Theory of Feature Learning
Noa Rubin, Kirsten Fischer, Javed Lindner +5
Feature learning in neural networks is crucial for their expressive power and inductive biases, motivating various theoretical approaches. Some approaches describe network behavior…
Critical feature learning in deep neural networks
Kirsten Fischer, Javed Lindner, David Dahmen +3
A key property of neural networks driving their success is their ability to learn features from data. Understanding feature learning from a theoretical viewpoint is an emerging fie…
Learning Interacting Theories from Data
Claudia Merger, Alexandre René, Kirsten Fischer +5
One challenge of physics is to explain how collective properties arise from microscopic interactions. Indeed, interactions form the building blocks of almost all physical theories…