14 citations · 39 across the 15 of their papers we have counts for
Showing 2024Show all
2 papers · 1 filter
stat.ML2024
Towards Understanding the Universality of Transformers for Next-Token Prediction
Michael E. Sander, Gabriel Peyré
Causal Transformers are trained to predict the next token for a given context. While it is widely accepted that self-attention is crucial for encoding the causal structure of seque…
stat.ML2024
How do Transformers perform In-Context Autoregressive Learning?
Michael E. Sander, Raja Giryes, Taiji Suzuki +2
Transformers have achieved state-of-the-art performance in language modeling tasks. However, the reasons behind their tremendous success are still unclear. In this paper, towards a…