1 citations · 1 across the 4 of their papers we have counts for
6 papers
Universal interpolation for deep residual self-attention networks
Sibylle Marcotte, Joan Bruna
Universal approximation is a necessary qualitative property of learning architectures to benefit from scaling laws. While it is generically verified on a variety of neural architec…
Intrinsic training dynamics of deep neural networks
Sibylle Marcotte, Gabriel Peyré, Rémi Gribonval
A fundamental challenge in the theory of deep learning is to understand whether gradient-based training can promote parameters belonging to certain lower-dimensional structures (e.…
Transformative or Conservative? Conservation laws for ResNets and Transformers
Sibylle Marcotte, Rémi Gribonval, Gabriel Peyré
While conservation laws in gradient flow training dynamics are well understood for (mostly shallow) ReLU and linear networks, their study remains largely unexplored for more practi…
Keep the Momentum: Conservation Laws beyond Euclidean Gradient Flows
Sibylle Marcotte, Rémi Gribonval, Gabriel Peyré
Conservation laws are well-established in the context of Euclidean gradient flow dynamics, notably for linear or ReLU neural network training. Yet, their existence and principles f…
Abide by the Law and Follow the Flow: Conservation Laws for Gradient Flows
Sibylle Marcotte, Rémi Gribonval, Gabriel Peyré
Understanding the geometric properties of gradient descent dynamics is a key ingredient in deciphering the recent success of very large machine learning models. A striking observat…
Fast Multiscale Diffusion on Graphs
Sibylle Marcotte, Amélie Barbe, Rémi Gribonval +4
Diffusing a graph signal at multiple scales requires computing the action of the exponential of several multiples of the Laplacian matrix. We tighten a bound on the approximation e…