Showing stat.MLShow all
3 papers · 1 filter
stat.ML2026
Transformers as Measure-Theoretic Associative Memory: A Statistical Perspective and Minimax Optimality
Ryotaro Kawata, Taiji Suzuki
Transformers excel through content-addressable retrieval and the ability to exploit contexts of, in principle, unbounded length. We recast associative memory at the level of probab…
stat.ML2024
State Space Models are Provably Comparable to Transformers in Dynamic Token Selection
Naoki Nishikawa, Taiji Suzuki
Deep neural networks based on state space models (SSMs) are attracting significant attention in sequence modeling since their computational cost is much smaller than that of Transf…
stat.ML2024
How do Transformers perform In-Context Autoregressive Learning?
Michael E. Sander, Raja Giryes, Taiji Suzuki +2
Transformers have achieved state-of-the-art performance in language modeling tasks. However, the reasons behind their tremendous success are still unclear. In this paper, towards a…