14 citations · 39 across the 15 of their papers we have counts for
4 papers · 1 filter
Clustering in Deep Stochastic Transformers
Lev Fedorov, Michaël E. Sander, Romuald Elie +2
Transformers have revolutionized deep learning across various domains but understanding the precise token dynamics remains a theoretical challenge. Existing theories of deep Transf…
Towards Understanding the Universality of Transformers for Next-Token Prediction
Michael E. Sander, Gabriel Peyré
Causal Transformers are trained to predict the next token for a given context. While it is widely accepted that self-attention is crucial for encoding the causal structure of seque…
How do Transformers perform In-Context Autoregressive Learning?
Michael E. Sander, Raja Giryes, Taiji Suzuki +2
Transformers have achieved state-of-the-art performance in language modeling tasks. However, the reasons behind their tremendous success are still unclear. In this paper, towards a…
Implicit regularization of deep residual networks towards neural ODEs
Pierre Marion, Yu-Han Wu, Michael E. Sander +1
Residual neural networks are state-of-the-art deep learning models. Their continuous-depth analog, neural ordinary differential equations (ODEs), are also widely used. Despite thei…