5 papers
Function graph transformers universally approximate operators between function spaces
Takashi Furuya, David Mis, Ivan DokmaniÄ +2
We study the approximation of nonlinear operators between function spaces by transformers. Our approach is to lift functions to measures supported on their graphs and leverage a re…
Training Infinitely Deep and Wide Transformers
Raphaël Barboni, Maarten V. de Hoop, Takashi Furuya +1
Transformers have become the dominant architecture in modern machine learning, yet the theoretical understanding of their training dynamics remains limited. This paper develops a r…
Transformers through the lens of support-preserving maps between measures
Takashi Furuya, Maarten V. de Hoop, Matti Lassas
Transformers are deep architectures that define ``in-context maps'' which enable predicting new tokens based on a given set of tokens (such as a prompt in NLP applications or a set…
Can neural operators always be continuously discretized?
Takashi Furuya, Michael Puthawala, Maarten V. de Hoop +1
We consider the problem of discretization of neural operators between Hilbert spaces in a general framework including skip connections. We focus on bijective neural operators throu…
Transformers are Universal In-context Learners
Takashi Furuya, Maarten V. de Hoop, Gabriel Peyré
Transformers are deep architectures that define "in-context mappings" which enable predicting new tokens based on a given set of tokens (such as a prompt in NLP applications or a s…