1 citations · 1 across the 8 of their papers we have counts for
11 papers
Transformers as Adaptive Kernel Learners across Tasks for In-Context Learning
Zhaiming Shen, Alexander Cloninger, Rongjie Lai +1
Transformers have emerged as a dominant architecture in modern machine learning, with in-context learning (ICL) enabling models to adapt to new tasks by conditioning on prompts, wi…
Neural Scaling Laws of Transformer Operator Network
Haoran Yan, Zhongjie Shi, Yuanzhe Xi +2
Transformers have emerged as powerful architectures for learning solution operators of physical systems. Empirically the prediction error has been observed to decrease when the dat…
Generalization Guarantees for Multi-Input Neural Operator Learning in Sobolev Spaces
Yahong Yang, Zecheng Zhang, Wei Zhu +2
We develop approximation and generalization error estimates for multi-input neural operators, with the output error measured in Sobolev norms. In contrast to standard operator-lear…
Understanding In-Context Learning for Nonlinear Regression with Transformers: Attention as Featurizer
Alexander Hsu, Zhaiming Shen, Wenjing Liao +1
Pre-trained transformers are able to learn from examples provided as part of the prompt without any weight updates, a remarkable ability known as in-context learning (ICL). Despite…
Data-Driven Model Reduction using WeldNet: Windowed Encoders for Learning Dynamics
Biraj Dahal, Jiahui Cheng, Hao Liu +2
Many problems in science and engineering involve time-dependent, high dimensional datasets arising from complex physical processes, which are costly to simulate. In this work, we p…
IDENT Review: Recent Advances in Identification of Differential Equations from Noisy Data
Roy Y. He, Hao Liu, Wenjing Liao +1
Differential equations and numerical methods are extensively used to model various real-world phenomena in science and engineering. With modern developments, we aim to find the und…
Identification of Differential Equations by Dynamics-Guided Weighted Weak Form with Voting
Jiahui Cheng, Sung Ha Kang, Haomin Zhou +1
In the identification of differential equations from data, significant progresses have been made with the weak/integral formulation. In this paper, we explore the direction of find…
Coefficient-to-Basis Network: A Fine-Tunable Operator Learning Framework for Inverse Problems with Adaptive Discretizations and Theoretical Guarantees
Zecheng Zhang, Hao Liu, Wenjing Liao +1
We propose a Coefficient-to-Basis Network (C2BNet), a novel framework for solving inverse problems within the operator learning paradigm. C2BNet efficiently adapts to different dis…
Neural Scaling Laws of Deep ReLU and Deep Operator Network: A Theoretical Study
Hao Liu, Zecheng Zhang, Wenjing Liao +1
Neural scaling laws play a pivotal role in the performance of deep neural networks and have been observed in a wide range of tasks. However, a complete theoretical framework for un…
Deep Neural Networks are Adaptive to Function Regularity and Data Distribution in Approximation and Estimation
Hao Liu, Jiahui Cheng, Wenjing Liao
Deep learning has exhibited remarkable results across diverse areas. To understand its success, substantial research has been directed towards its theoretical foundations. Neverthe…
Generalization Error Guaranteed Auto-Encoder-Based Nonlinear Model Reduction for Operator Learning
Hao Liu, Biraj Dahal, Rongjie Lai +1
Many physical processes in science and engineering are naturally represented by operators between infinite-dimensional function spaces. The problem of operator learning, in this co…