2 citations · 4 across the 8 of their papers we have counts for
4 papers · 1 filter
Scaling Limits of Long-Context Transformers
Giuseppe Bruno, Shi Chen, Zhengjiang Lin +2
We study the long-context limit of softmax self-attention with a fixed query and a random context of i.i.d. keys on the sphere, viewing the inverse temperature as the sca…
A multiscale analysis of mean-field transformers in the moderate interaction regime
Giuseppe Bruno, Federico Pasqualotto, Andrea Agazzi
In this paper, we study the evolution of tokens through the depth of encoder-only transformer models at inference time by modeling them as a system of particles interacting in a me…
Emergence of meta-stable clustering in mean-field transformer models
Giuseppe Bruno, Federico Pasqualotto, Andrea Agazzi
We model the evolution of tokens within a deep stack of Transformer layers as a continuous-time flow on the unit sphere, governed by a mean-field interacting particle system, build…
The Challenges of the Nonlinear Regime for Physics-Informed Neural Networks
Andrea Bonfanti, Giuseppe Bruno, Cristina Cipriani
The Neural Tangent Kernel (NTK) viewpoint is widely employed to analyze the training dynamics of overparameterized Physics-Informed Neural Networks (PINNs). However, unlike the cas…