6 papers
Training-Free Universal Approximation by Prompting Random Transformers
Alexander Hsu, Rongjie Lai
How expressive is prompting a transformer? Answering this question is important for separating the roles of prompting, architecture, and pretraining in transformer models, and for…
Understanding In-Context Learning on Structured Manifolds: Bridging Attention to Kernel Methods
Zhaiming Shen, Alexander Hsu, Rongjie Lai +1
While in-context learning (ICL) has achieved remarkable success in natural language and vision domains, its theoretical understanding-particularly in the context of structured geom…
Understanding In-Context Learning for Nonlinear Regression with Transformers: Attention as Featurizer
Alexander Hsu, Zhaiming Shen, Wenjing Liao +1
Pre-trained transformers are able to learn from examples provided as part of the prompt without any weight updates, a remarkable ability known as in-context learning (ICL). Despite…
A joint optimization approach to identifying sparse dynamics using least squares kernel collocation
Alexander W. Hsu, Ike Griss Salas, Jacob M. Stevens-Haas +3
We develop an all-at-once modeling framework for learning systems of ordinary differential equations (ODE) from scarce, partial, and noisy observations of the states. The proposed…
Operator Learning at Machine Precision
Aras Bacho, Aleksei G. Sorokin, Xianjin Yang +6
Neural operator learning methods have garnered significant attention in scientific computing for their ability to approximate infinite-dimensional operators. However, increasing th…
Data-Efficient Kernel Methods for Learning Differential Equations and Their Solution Operators: Algorithms and Error Analysis
Yasamin Jalalian, Juan Felipe Osorio Ramirez, Alexander Hsu +2
We introduce a novel kernel-based framework for learning differential equations and their solution maps that is efficient in data requirements, in terms of solution examples and am…