3 citations · 10 across the 27 of their papers we have counts for
30 papers
PICKLES: Covariance-Preconditioned Kinetic Langevin Ensemble Sampler
Hong Ye Tan, Stanley Osher, Yifan Chen
We propose a covariance-preconditioned kinetic Langevin ensemble sampler named PICKLES. This is a kinetic analogue of the affine invariant interacting Langevin dynamics (ALDI). PIC…
Vocabulary-size-independent Convergence of Discrete Diffusion Models: adjoint equations induce the right space
Kelvin Kan, Xingjian Li, Benjamin J. Zhang +3
Discrete diffusion has become a leading framework for generative modeling in various applications including language, vision, and biology. Existing convergence theory, however, exh…
Scalable Fixed-Point Framework for High-Dimensional Hamilton-Jacobi Equations
Yesom Park, Stanley Osher
We propose a novel, mesh-free, and gradient-free fixed-point approach for computing viscosity solutions of high-dimensional Hamilton-Jacobi (HJ) equations. By leveraging the Hopf-L…
SymPlex: A Structure-Aware Transformer for Symbolic PDE Solving
Yesom Park, Annie C. Lu, Shao-Ching Huang +3
We propose SymPlex, a reinforcement learning framework for discovering analytical symbolic solutions to partial differential equations (PDEs) without access to ground-truth express…
Accelerated Regularized Wasserstein Proximal Sampling Algorithms
Hong Ye Tan, Stanley Osher, Wuchen Li
We consider sampling from a Gibbs distribution by evolving a finite number of particles using a particular score estimator rather than Brownian motion. To accelerate the particles,…
Dynamical Implicit Neural Representations
Yesom Park, Kelvin Kan, Thomas Flynn +4
Implicit Neural Representations (INRs) provide a powerful continuous framework for modeling complex visual and geometric signals, but spectral bias remains a fundamental challenge,…
Sparse Transformer Architectures via Regularized Wasserstein Proximal Operator with Prior
Fuqun Han, Stanley Osher, Wuchen Li
In this work, we propose a sparse transformer architecture that incorporates prior information about the underlying data distribution directly into the transformer structure of the…
Stability of Transformers under Layer Normalization
Kelvin Kan, Xingjian Li, Benjamin J. Zhang +4
Despite their widespread use, training deep Transformers can be unstable. Layer normalization, a standard component, improves training stability, but its placement has often been a…
Zero-Shot Transferable Solution Method for Parametric Optimal Control Problems
Xingjian Li, Kelvin Kan, Deepanshu Verma +3
This paper presents a transferable solution method for optimal control problems with varying objectives using function encoder (FE) policies. Traditional optimization-based approac…
Preconditioned Regularized Wasserstein Proximal Sampling
Hong Ye Tan, Stanley Osher, Wuchen Li
We consider sampling from a Gibbs distribution by evolving finitely many particles. We propose a preconditioned version of a recently proposed noise-free sampling method, governed…
Simulating Fokker-Planck equations via mean field control of score-based normalizing flows
Mo Zhou, Stanley Osher, Wuchen Li
The Fokker--Planck (FP) equation governs the evolution of densities for stochastic dynamics of physical systems, such as the Langevin dynamics and the Lorenz system. This work simu…
Optimal Control for Transformer Architectures: Enhancing Generalization, Robustness and Efficiency
Kelvin Kan, Xingjian Li, Benjamin J. Zhang +3
We study Transformers through the perspective of optimal control theory, using tools from continuous-time formulations to derive actionable insights into training and architecture…
Splitting Regularized Wasserstein Proximal Algorithms for Nonsmooth Sampling Problems
Fuqun Han, Stanley Osher, Wuchen Li
Sampling from nonsmooth target probability distributions is essential in various applications, including the Bayesian Lasso. We propose a splitting-based sampling algorithm for the…
Neural Implicit Solution Formula for Efficiently Solving Hamilton-Jacobi Equations
Yesom Park, Stanley Osher
This paper presents an implicit solution formula for the Hamilton-Jacobi partial differential equation (HJ PDE). The formula is derived using the method of characteristics and is s…
OT-Transformer: A Continuous-time Transformer Architecture with Optimal Transport Regularization
Kelvin Kan, Xingjian Li, Stanley Osher
Transformers have achieved state-of-the-art performance in numerous tasks. In this paper, we propose a continuous-time formulation of transformers. Specifically, we consider a dyna…
Inexact Proximal Point Algorithms for Zeroth-Order Global Optimization
Minxin Zhang, Fuqun Han, Yat Tin Chow +2
This work concerns the zeroth-order global minimization of continuous nonconvex functions with a unique global minimizer and possibly multiple local minimizers. We formulate a theo…
VICON: Vision In-Context Operator Networks for Multi-Physics Fluid Dynamics Prediction
Yadi Cao, Yuxuan Liu, Liu Yang +3
In-Context Operator Networks (ICONs) have demonstrated the ability to learn operators across diverse partial differential equations using few-shot, in-context learning. However, ex…
A Natural Primal-Dual Hybrid Gradient Method for Adversarial Neural Network Training on Solving Partial Differential Equations
Shu Liu, Stanley Osher, Wuchen Li
We propose a scalable preconditioned primal-dual hybrid gradient algorithm for solving partial differential equations (PDEs). We multiply the PDE with a dual test function to obtai…
Gradient-adjusted underdamped Langevin dynamics for sampling
Xinzhe Zuo, Stanley Osher, Wuchen Li
Sampling from a target distribution is a fundamental problem. Traditional Markov chain Monte Carlo (MCMC) algorithms, such as the unadjusted Langevin algorithm (ULA), derived from…
Score-based Neural Ordinary Differential Equations for Computing Mean Field Control Problems
Mo Zhou, Stanley Osher, Wuchen Li
Classical neural ordinary differential equations (ODEs) are powerful tools for approximating the log-density functions in high-dimensional spaces along trajectories, where neural n…
Convergence of Noise-Free Sampling Algorithms with Regularized Wasserstein Proximals
Fuqun Han, Stanley Osher, Wuchen Li
In this work, we investigate the convergence properties of the backward regularized Wasserstein proximal (BRWP) method for sampling a target distribution. The BRWP approach can be…
Efficient Computation of Mean field Control based Barycenters from Reaction-Diffusion Systems
Arjun Vijaywargiya, Guosheng Fu, Stanley Osher +1
We develop a class of barycenter problems based on mean field control problems in three dimensions with associated reactive-diffusion systems of unnormalized multi-species densitie…
A Primal-dual hybrid gradient method for solving optimal control problems and the corresponding Hamilton-Jacobi PDEs
Tingwei Meng, Siting Liu, Wuchen Li +1
Optimal control problems are crucial in various domains, including path planning, robotics, and humanoid control, demonstrating their broad applicability. The connection between op…
Numerical Analysis on Neural Network Projected Schemes for Approximating One Dimensional Wasserstein Gradient Flows
Xinzhe Zuo, Jiaxi Zhao, Shu Liu +2
We provide a numerical analysis and computation of neural network projected schemes for approximating one dimensional Wasserstein gradient flows. We approximate the Lagrangian mapp…