12 citations · 23 across the 9 of their papers we have counts for
10 papers
A Closed Form Depth-Width Law for Neural Collapse in Deep Linear Networks
Pankhil Gawade, Kirsten Fischer, Guy Wolf +1
How depth and width of neural networks shape learned representations is usually studied only after training. We show that, in a tractable feature-learning setting, this dependence…
Phase Transitions in Attention: A Bayesian Theory of Copy Head Emergence
Itay Lavie, Kirsten Fischer, Andrey Lekov +3
Attention is the key mechanism underlying in-context learning in transformers, and attention patterns have been observed empirically to emerge abruptly during training. We present…
A unified theory of feature learning in RNNs and DNNs
Jan P. Bauer, Kirsten Fischer, Moritz Helias +1
Recurrent and deep neural networks (RNNs/DNNs) are cornerstone architectures in machine learning. Remarkably, RNNs differ from DNNs only by weight sharing, as can be shown through…
Characterizing Neural Manifolds' Properties and Curvatures using Normalizing Flows
Peter Bouss, Sandra Nestler, Kirsten Fischer +3
Neuronal activity is found to lie on low-dimensional manifolds embedded within the high-dimensional neuron space. Variants of principal component analysis are frequently employed t…
From Kernels to Features: A Multi-Scale Adaptive Theory of Feature Learning
Noa Rubin, Kirsten Fischer, Javed Lindner +5
Feature learning in neural networks is crucial for their expressive power and inductive biases, motivating various theoretical approaches. Some approaches describe network behavior…
Critical feature learning in deep neural networks
Kirsten Fischer, Javed Lindner, David Dahmen +3
A key property of neural networks driving their success is their ability to learn features from data. Understanding feature learning from a theoretical viewpoint is an emerging fie…
Learning Interacting Theories from Data
Claudia Merger, Alexandre René, Kirsten Fischer +5
One challenge of physics is to explain how collective properties arise from microscopic interactions. Indeed, interactions form the building blocks of almost all physical theories…
Field theory for optimal signal propagation in ResNets
Kirsten Fischer, David Dahmen, Moritz Helias
Residual networks have significantly better trainability and thus performance than feed-forward networks at large depth. Introducing skip connections facilitates signal propagation…
Decomposing neural networks as mappings of correlation functions
Kirsten Fischer, Alexandre René, Christian Keup +3
Understanding the functional principles of information processing in deep neural networks continues to be a challenge, in particular for networks with trained and thus non-random w…
Meta-Learning Acquisition Functions for Transfer Learning in Bayesian Optimization
Michael Volpp, Lukas P. Fröhlich, Kirsten Fischer +4
Transferring knowledge across tasks to improve data-efficiency is one of the open key challenges in the field of global black-box optimization. Readily available algorithms are typ…