34 citations · 34 across the 3 of their papers we have counts for
4 papers
Kernel-based Interpolatory Methods for Context Compression of Attention Mechanisms
Michel Fabrice Serret, Laura Grigori
The bottleneck of attention mechanisms, which are the fundamental building block of modern language models, is their quadratic complexity in terms of context size. There has thus b…
Attention Mechanisms Through the Lens of Numerical Methods: Approximation Methods and Alternative Formulations
Michel Fabrice Serret, Alice Cortinovis, Yijun Dong +10
The attention mechanism is the computational core of modern Transformer architectures, but its quadratic complexity in the input sequence length is the bottleneck for large-scale i…
Understanding Transformers and Attention Mechanisms: An Introduction for Applied Mathematicians
Michel Fabrice Serret
This document provides a brief introduction to the attention mechanism used in modern language models based on the Transformer architecture. We first illustrate how text is encoded…
Solving optimization problems with Rydberg analog quantum computers: Realistic requirements for quantum advantage using noisy simulation and classical benchmarks
Michel Fabrice Serret, Bertrand Marchand, Thomas Ayral
Platforms of Rydberg atoms have been proposed as promising candidates to solve some combinatorial optimization problems. Here, we compute quantitative requirements on the system si…