11 papers
Self-supervised In-context Operator Learning for Stochastic Mean-Field Control
Suyi Gao, Mo Zhou, Rongjie Lai
Stochastic mean-field control (MFC) provides a fundamental framework for coordinating large populations of interacting agents under uncertainty, with a wide range of applications.…
Training-Free Universal Approximation by Prompting Random Transformers
Alexander Hsu, Rongjie Lai
How expressive is prompting a transformer? Answering this question is important for separating the roles of prompting, architecture, and pretraining in transformer models, and for…
Neural operator learning for collision-aware trajectory planning of spacecraft swarms
Sidhdharth D. Sikka, Suyi Gao, Zehui Lu +2
Autonomous spacecraft swarms must plan fuel-efficient, collision-free maneuvers in increasingly congested orbits, yet classical trajectory optimization scales poorly as pairwise sa…
Understanding In-Context Learning on Structured Manifolds: Bridging Attention to Kernel Methods
Zhaiming Shen, Alexander Hsu, Rongjie Lai +1
While in-context learning (ICL) has achieved remarkable success in natural language and vision domains, its theoretical understanding-particularly in the context of structured geom…
Transformers for Learning on Noisy and Task-Level Manifolds: Approximation and Generalization Insights
Zhaiming Shen, Alex Havrilla, Rongjie Lai +2
Transformers serve as the foundational architecture for large language and video generation models, such as GPT, BERT, SORA and their successors. Empirical studies have demonstrate…
Understanding In-Context Learning for Nonlinear Regression with Transformers: Attention as Featurizer
Alexander Hsu, Zhaiming Shen, Wenjing Liao +1
Pre-trained transformers are able to learn from examples provided as part of the prompt without any weight updates, a remarkable ability known as in-context learning (ICL). Despite…