4 papers
A Mean-Field Theory of Transformers: Well-Posedness of the Coupled Data--Parameter Dynamics and Global Convergence of Training
Michael Herty, Hailiang Liu
We develop a rigorous mean-field theory for transformer networks that captures two large-scale limits inherent in the architecture: the number of tokens in the input s…
Inf-Sup Neural Networks for High Dimensional PDEs
Ziren Chen, Hailiang Liu
Solving partial differential equations (PDEs) in high dimensions remains challenging due to the curse of dimensionality. We propose a neural-network-based framework that reformulat…
Neural solver for Wasserstein Geodesics and optimal transport dynamics
Hailiang Liu, Yan-Han Chen
In recent years, the machine learning community has increasingly embraced the optimal transport (OT) framework for modeling distributional relationships. In this work, we introduce…
Global Convergence in Neural ODEs: Impact of Activation Functions
Tianxiang Gao, Siyuan Sun, Hailiang Liu +1
Neural Ordinary Differential Equations (ODEs) have been successful in various applications due to their continuous nature and parameter-sharing efficiency. However, these unique ch…