305 citations · 1.3k across the 52 of their papers we have counts for
55 papers
Teacher Geometry Shapes Learnability in Teacher-Student Networks
Kai J. Sandbrink, Flavio Martinelli, Alexander van Meegen +2
Teacher-student systems, in which a teacher neural network generates training labels so that a student neural network can learn to implement the same function, are widely used as a…
Self-supervised local learning rules learn the hidden hierarchical structure of high-dimensional data
Ariane Delrocq, Wu S. Zihan, Guillaume Bellec +1
The brain learns abstract representations of high-dimensional sensory input, but the plasticity rules that enable such learning are unknown. We study biologically plausible algorit…
Can Local Learning Match Self-Supervised Backpropagation?
Wu S. Zihan, Ariane Delrocq, Wulfram Gerstner +1
While end-to-end self-supervised learning with backpropagation (global BP-SSL) has become central for training modern AI systems, theories of local self-supervised learning (local-…
Data Augmentation Techniques to Reverse-Engineer Neural Network Weights from Input-Output Queries
Alexander Beiser, Flavio Martinelli, Wulfram Gerstner +1
Network weights can be reverse-engineered given enough informative samples of a network's input-output function. In a teacher-student setup, this translates into collecting a datas…
Flat Channels to Infinity in Neural Loss Landscapes
Flavio Martinelli, Alexander Van Meegen, Berfin Şimşek +2
The loss landscapes of neural networks contain minima and saddle points that may be connected in flat regions or appear in isolation. We identify and characterize a special structu…
Should Under-parameterized Student Networks Copy or Average Teacher Weights?
Berfin Şimşek, Amire Bendjeddou, Wulfram Gerstner +1
Any continuous function can be approximated arbitrarily well by a neural network with sufficiently many neurons . We consider the case when itself is a neural networ…