175 citations · 317 across the 46 of their papers we have counts for
50 papers
Loss Curvature as a Sampling-Free Proxy for Memorization in Diffusion Models
Shashank Sule, Xinyue Yu, Boonam Shin +3
Understanding the mechanism behind why diffusion models memorize their training data remains an open question in generative modeling. In this paper, we examine this question from t…
Wasserstein Moment Nudging for Vlasov-Poisson Data Assimilation
Liyao Lyu, Xinyue Yu, David Schneidinger +1
We introduce a continuous data assimilation method for particle-in-cell simulations of the Vlasov-Poisson equation when only hydrodynamic moments are observed. The forecast state i…
Kernel Methods for Learning Operators with Multiple Inputs and Outputs
Adrien Weihs, Chunyang Liao, Jingmin Sun +1
Learning mappings between infinite-dimensional objects is a central challenge in scientific machine learning. We introduce a general kernel-based encoder-decoder framework for oper…
Multiscale Nudging: From Macroscopic Observations to Microscopic Dynamics
Liyao Lyu, Xinyue Yu, Hayden Schaeffer
We introduce a measure-based nudging framework for assimilating macroscopic observations into microscopic mean-field particle dynamics. The central difficulty is a representation m…
Multiple Neural Operators Achieve Near-Optimal Rates for Multi-Task Learning
Adrien Weihs, Hayden Schaeffer
We study the approximation and statistical complexity of learning collections of operators in a shared multi-task setting, with a focus on the Multiple Neural Operators (MNO) archi…
MVNN: A Measure-Valued Neural Network for Learning McKean-Vlasov Dynamics from Particle Data
Liyao Lyu, Xinyue Yu, Hayden Schaeffer
Collective behaviors that emerge from interactions are fundamental to numerous biological systems. To learn such interacting forces from observations, we introduce a measure-valued…
Generalization Bounds and Statistical Guarantees for Multi-Task and Multiple Operator Learning with MNO Networks
Adrien Weihs, Hayden Schaeffer
Multiple operator learning concerns learning operator families indexed by an operator descriptor . Training data are collected hierarchically by sampl…
Adam Improves Muon: Adaptive Moment Estimation with Orthogonalized Momentum
Minxin Zhang, Yuxuan Liu, Hayden Schaeffer
Efficient stochastic optimization typically integrates an update direction that performs well in the deterministic regime with a mechanism adapting to stochastic perturbations. Whi…
PI-MFM: Physics-informed multimodal foundation model for solving partial differential equations
Min Zhu, Jingmin Sun, Zecheng Zhang +2
Partial differential equations (PDEs) govern a wide range of physical systems, and recent multimodal foundation models have shown promise for learning PDE solution operators across…
Regularized Random Fourier Features and Finite Element Reconstruction for Operator Learning in Sobolev Space
Xinyue Yu, Hayden Schaeffer
Operator learning is a data-driven approximation of mappings between infinite-dimensional function spaces, such as the solution operators of partial differential equations. Kernel-…
Finite Element Representation Network (FERN) for Operator Learning with a Localized Trainable Basis
Zecheng Zhang, Hao Liu, Guosheng Fu +2
We propose a finite-element local basis-based operator learning framework for solving partial differential equations (PDEs). Operator learning aims to approximate mappings from inp…
A Deep Learning Framework for Multi-Operator Learning: Architectures and Approximation Theory
Adrien Weihs, Jingmin Sun, Zecheng Zhang +1
While many problems in machine learning focus on learning mappings between finite-dimensional spaces, scientific applications require approximating mappings between function spaces…
AdaGrad Meets Muon: Adaptive Stepsizes for Orthogonal Updates
Minxin Zhang, Yuxuan Liu, Hayden Schaeffer
The recently proposed Muon optimizer updates weight matrices via orthogonalized momentum and has demonstrated strong empirical success in large language model training. However, it…
Cauchy Random Features for Operator Learning in Sobolev Space
Chunyang Liao, Deanna Needell, Hayden Schaeffer
Operator learning is the approximation of operators between infinite dimensional Banach spaces using machine learning approaches. While most progress in this area has been driven b…
A Multimodal PDE Foundation Model for Prediction and Scientific Text Descriptions
Elisa Negrini, Yuxuan Liu, Liu Yang +2
Neural networks are one tool for approximating non-linear differential equations used in scientific computing tasks such as surrogate modeling, real-time predictions, and optimal c…
BCAT: A Block Causal Transformer for PDE Foundation Models for Fluid Dynamics
Yuxuan Liu, Jingmin Sun, Hayden Schaeffer
We introduce BCAT, a PDE foundation model designed for autoregressive prediction of solutions to two dimensional fluid dynamics problems. Our approach uses a block causal transform…
Inexact Proximal Point Algorithms for Zeroth-Order Global Optimization
Minxin Zhang, Fuqun Han, Yat Tin Chow +2
This work concerns the zeroth-order global minimization of continuous nonconvex functions with a unique global minimizer and possibly multiple local minimizers. We formulate a theo…
Differentially Private Random Feature Model
Chunyang Liao, Deanna Needell, Hayden Schaeffer +1
Designing privacy-preserving machine learning algorithms has received great attention in recent years, especially in the setting when the data contains sensitive information. Diffe…
DeepONet as a Multi-Operator Extrapolation Model: Distributed Pretraining with Physics-Informed Fine-Tuning
Zecheng Zhang, Christian Moya, Lu Lu +2
We propose a novel fine-tuning method to achieve multi-operator learning through training a distributed neural operator with diverse function data and then zero-shot fine-tuning th…
VICON: Vision In-Context Operator Networks for Multi-Physics Fluid Dynamics Prediction
Yadi Cao, Yuxuan Liu, Liu Yang +3
In-Context Operator Networks (ICONs) have demonstrated the ability to learn operators across diverse partial differential equations using few-shot, in-context learning. However, ex…
Neural Scaling Laws of Deep ReLU and Deep Operator Network: A Theoretical Study
Hao Liu, Zecheng Zhang, Wenjing Liao +1
Neural scaling laws play a pivotal role in the performance of deep neural networks and have been observed in a wide range of tasks. However, a complete theoretical framework for un…
Time-Series Forecasting, Knowledge Distillation, and Refinement within a Multimodal PDE Foundation Model
Derek Jollie, Jingmin Sun, Zecheng Zhang +1
Symbolic encoding has been used in multi-operator learning as a way to embed additional information for distinct time-series data. For spatiotemporal systems described by time-depe…
PROSE-FD: A Multimodal PDE Foundation Model for Learning Multiple Operators for Forecasting Fluid Dynamics
Yuxuan Liu, Jingmin Sun, Xinjie He +3
We propose PROSE-FD, a zero-shot multimodal PDE foundational model for simultaneous prediction of heterogeneous two-dimensional physical systems related to distinct fluid dynamics…
LeMON: Learning to Learn Multi-Operator Networks
Jingmin Sun, Zecheng Zhang, Hayden Schaeffer
Single-operator learning involves training a deep neural network to learn a specific operator, whereas recent work in multi-operator learning uses an operator embedding structure t…