3 citations · 10 across the 27 of their papers we have counts for
10 papers · 1 filter
Dynamical Implicit Neural Representations
Yesom Park, Kelvin Kan, Thomas Flynn +4
Implicit Neural Representations (INRs) provide a powerful continuous framework for modeling complex visual and geometric signals, but spectral bias remains a fundamental challenge,…
Sparse Transformer Architectures via Regularized Wasserstein Proximal Operator with Prior
Fuqun Han, Stanley Osher, Wuchen Li
In this work, we propose a sparse transformer architecture that incorporates prior information about the underlying data distribution directly into the transformer structure of the…
Stability of Transformers under Layer Normalization
Kelvin Kan, Xingjian Li, Benjamin J. Zhang +4
Despite their widespread use, training deep Transformers can be unstable. Layer normalization, a standard component, improves training stability, but its placement has often been a…
Zero-Shot Transferable Solution Method for Parametric Optimal Control Problems
Xingjian Li, Kelvin Kan, Deepanshu Verma +3
This paper presents a transferable solution method for optimal control problems with varying objectives using function encoder (FE) policies. Traditional optimization-based approac…
Preconditioned Regularized Wasserstein Proximal Sampling
Hong Ye Tan, Stanley Osher, Wuchen Li
We consider sampling from a Gibbs distribution by evolving finitely many particles. We propose a preconditioned version of a recently proposed noise-free sampling method, governed…
Simulating Fokker-Planck equations via mean field control of score-based normalizing flows
Mo Zhou, Stanley Osher, Wuchen Li
The Fokker--Planck (FP) equation governs the evolution of densities for stochastic dynamics of physical systems, such as the Langevin dynamics and the Lorenz system. This work simu…
Optimal Control for Transformer Architectures: Enhancing Generalization, Robustness and Efficiency
Kelvin Kan, Xingjian Li, Benjamin J. Zhang +3
We study Transformers through the perspective of optimal control theory, using tools from continuous-time formulations to derive actionable insights into training and architecture…
Splitting Regularized Wasserstein Proximal Algorithms for Nonsmooth Sampling Problems
Fuqun Han, Stanley Osher, Wuchen Li
Sampling from nonsmooth target probability distributions is essential in various applications, including the Bayesian Lasso. We propose a splitting-based sampling algorithm for the…
Neural Implicit Solution Formula for Efficiently Solving Hamilton-Jacobi Equations
Yesom Park, Stanley Osher
This paper presents an implicit solution formula for the Hamilton-Jacobi partial differential equation (HJ PDE). The formula is derived using the method of characteristics and is s…
OT-Transformer: A Continuous-time Transformer Architecture with Optimal Transport Regularization
Kelvin Kan, Xingjian Li, Stanley Osher
Transformers have achieved state-of-the-art performance in numerous tasks. In this paper, we propose a continuous-time formulation of transformers. Specifically, we consider a dyna…