26 citations · 110 across the 25 of their papers we have counts for
29 papers
Which Geometry on Which Layer? Measured Sign-Spectral Assignment in Mixed-Optimizer Training
Hiroki Naganuma, Laura Gomezjurado, Mahdi Ghaznavi +5
Modern neural-network training uses mixed recipes, assigning AdamW-like or sign-based updates to embedding and output layers and spectral updates to hidden matrices. These assignme…
Finite-Sample Approximation of Hessian-Guided Perturbed Wasserstein Gradient Flows
Ryotaro Kawata, Atsushi Nitanda, Taiji Suzuki
Wasserstein gradient flow extends gradient descent to probability measures. Its Hessian-guided perturbed variant (PWGF) adds Gaussian perturbations to escape saddle points in nonco…
Learning Decision-Stump Thresholds in Context: Dynamics of Softmax Attention
Hong Ha Le, Jackie Lok, Atsushi Nitanda +1
Estimating a decision threshold requires locating observations near an unknown boundary. We study how gradient-based pretraining learns this statistical rule in a two-parameter sof…
Diffusion Transformers are Provably Optimal In-context Generators
Guoji Fu, Tomoya Wakayama, Ryotaro Kawata +3
Generative foundation models are attracting interest for their ability to produce desired outputs from demonstrations given at inference time, without updating parameters. However,…
Intrinsic Wasserstein Rates for Score-Based Generative Models on Smooth Manifolds
Guoji Fu, Taiji Suzuki, Wee Sun Lee +1
Score-based generative models are trained in high-dimensional ambient spaces, yet many data distributions are supported on low-dimensional nonlinear structures. We prove that, for…
DPRM: A Plug-in Doob h transform-induced Token-Ordering Module for Discrete Diffusion Models
Dake Bu, Wei Huang, Andi Han +5
Discrete diffusion models admit many token orders, yet most systems rely on confidence-based decoding. Confidence is a strong and efficient heuristic, but it can be myopic because…