2 citations · 2 across the 5 of their papers we have counts for
8 papers
Stability of Measure-to-Measure Transformers on Sub-Gaussian Data
Frank Cole, Nicholas H. Nelsen, Takashi Furuya
Transformers have exhibited impressive empirical success across various domains, but their theoretical foundations remain less developed. This work constitutes a mathematical study…
Universal Approximation of Measure-to-Measure Operators by Pushforwards
Takashi Furuya, Nicholas H. Nelsen, Frank Cole
Many learning tasks map an input distribution to an output distribution. A natural way to model such an operator is to transform each input sample using a continuous function that…
Universality and Generalization of Causal Transformers Across Context Lengths
Takashi Furuya, Maarten V. de Hoop, Gabriel Peyré
Long contexts are central to modern transformer systems, but most expressivity results choose a different network for each fixed sequence length. We study whether one masked transf…
Function graph transformers universally approximate operators between function spaces
Takashi Furuya, David Mis, Ivan Dokmanić +2
We study the approximation of nonlinear operators between function spaces by transformers. Our approach is to lift functions to measures supported on their graphs and leverage a re…
Training Infinitely Deep and Wide Transformers
Raphaël Barboni, Maarten V. de Hoop, Takashi Furuya +1
Transformers have become the dominant architecture in modern machine learning, yet the theoretical understanding of their training dynamics remains limited. This paper develops a r…
Transformers through the lens of support-preserving maps between measures
Takashi Furuya, Maarten V. de Hoop, Matti Lassas
Transformers are deep architectures that define ``in-context maps'' which enable predicting new tokens based on a given set of tokens (such as a prompt in NLP applications or a set…
Can neural operators always be continuously discretized?
Takashi Furuya, Michael Puthawala, Maarten V. de Hoop +1
We consider the problem of discretization of neural operators between Hilbert spaces in a general framework including skip connections. We focus on bijective neural operators throu…
Transformers are Universal In-context Learners
Takashi Furuya, Maarten V. de Hoop, Gabriel Peyré
Transformers are deep architectures that define "in-context mappings" which enable predicting new tokens based on a given set of tokens (such as a prompt in NLP applications or a s…