5 papers
Transformers as Adaptive Kernel Learners across Tasks for In-Context Learning
Zhaiming Shen, Alexander Cloninger, Rongjie Lai +1
Transformers have emerged as a dominant architecture in modern machine learning, with in-context learning (ICL) enabling models to adapt to new tasks by conditioning on prompts, wi…
Scalable Graph Coreset Selection via Greedy Sampling
Zhaiming Shen, Alexander Cloninger
Sampling representative nodes from large graphs is fundamental to graph signal processing and network analysis, yet existing methods require access to the full graph Laplacian, mak…
Understanding In-Context Learning for Nonlinear Regression with Transformers: Attention as Featurizer
Alexander Hsu, Zhaiming Shen, Wenjing Liao +1
Pre-trained transformers are able to learn from examples provided as part of the prompt without any weight updates, a remarkable ability known as in-context learning (ICL). Despite…
Understanding In-Context Learning on Structured Manifolds: Bridging Attention to Kernel Methods
Zhaiming Shen, Alexander Hsu, Rongjie Lai +1
While in-context learning (ICL) has achieved remarkable success in natural language and vision domains, its theoretical understanding-particularly in the context of structured geom…
Transformers for Learning on Noisy and Task-Level Manifolds: Approximation and Generalization Insights
Zhaiming Shen, Alex Havrilla, Rongjie Lai +2
Transformers serve as the foundational architecture for large language and video generation models, such as GPT, BERT, SORA and their successors. Empirical studies have demonstrate…