1 citations · 1 across the 9 of their papers we have counts for
6 papers · 1 filter
Stability of Transformers under Layer Normalization
Kelvin Kan, Xingjian Li, Benjamin J. Zhang +4
Despite their widespread use, training deep Transformers can be unstable. Layer normalization, a standard component, improves training stability, but its placement has often been a…
Probabilistic operator learning: generative modeling and uncertainty quantification for foundation models of differential equations
Benjamin J. Zhang, Siting Liu, Stanley J. Osher +1
In-context operator networks (ICON) are a class of operator learning methods based on the novel architectures of foundation models. Trained on a diverse set of datasets of initial…
ExAct: A Video-Language Benchmark for Expert Action Analysis
Han Yi, Yulu Pan, Feihong He +4
We present ExAct, a new video-language benchmark for expert-level understanding of skilled physical human activities. Our new benchmark contains 3521 expert-curated video question-…
Particle exchange Monte Carlo methods for eigenfunction and related nonlinear problems
Paul Dupuis, Benjamin J. Zhang
We introduce and develop a novel particle exchange Monte Carlo method. Whereas existing methods apply to eigenfunction problems where the eigenvalue is known (e.g., integrals with…
Optimal Control for Transformer Architectures: Enhancing Generalization, Robustness and Efficiency
Kelvin Kan, Xingjian Li, Benjamin J. Zhang +3
We study Transformers through the perspective of optimal control theory, using tools from continuous-time formulations to derive actionable insights into training and architecture…
Proximal optimal transport divergences
Ricardo Baptista, Panagiota Birmpa, Markos A. Katsoulakis +2
We introduce the proximal optimal transport divergence, a novel discrepancy measure that interpolates between information divergences and optimal transport distances via an infimal…