17 citations · 29 across the 10 of their papers we have counts for
11 papers
Flat Channels to Infinity in Neural Loss Landscapes
Flavio Martinelli, Alexander Van Meegen, Berfin Şimşek +2
The loss landscapes of neural networks contain minima and saddle points that may be connected in flat regions or appear in isolation. We identify and characterize a special structu…
Learning Gaussian Multi-Index Models with Gradient Flow: Time Complexity and Directional Convergence
Berfin Şimşek, Amire Bendjeddou, Daniel Hsu
This work focuses on the gradient flow dynamics of a neural network model that uses correlation loss to approximate a multi-index function on high-dimensional standard Gaussian dat…
Learning Associative Memories with Gradient Descent
Vivien Cabannes, Berfin Simsek, Alberto Bietti
This work focuses on the training dynamics of one associative memory module storing outer products of token embeddings. We reduce this problem to the study of a system of particles…
Loss Landscape of Shallow ReLU-like Neural Networks: Stationary Points, Saddle Escape, and Network Embedding
Frank Zhengqing Wu, Berfin Simsek, Francois Gaston Ged
In this paper, we study the loss landscape of one-hidden-layer neural networks with ReLU-like activation functions trained with the empirical squared loss using gradient descent (G…
Should Under-parameterized Student Networks Copy or Average Teacher Weights?
Berfin Şimşek, Amire Bendjeddou, Wulfram Gerstner +1
Any continuous function can be approximated arbitrarily well by a neural network with sufficiently many neurons . We consider the case when itself is a neural networ…
Statistical physics, Bayesian inference and neural information processing
Erin Grant, Sandra Nestler, Berfin Şimşek +1
Lecture notes from the course given by Professor Sara A. Solla at the Les Houches summer school on "Statistical physics of Machine Learning". The notes discuss neural information p…