9 papers
On the Complexity of Finding Stationary Points of p-th Order Hölder Smooth Functions in One Dimension
Huanjian Zhou, Taiji Suzuki
We study the query complexity of finding an an -stationary points of a one-dimensional non-convex function whose -th derivative is -Hölder continuous. We deter…
Steepest Guidance: A Practical and Principled Approach to Inference-Time Alignment of Flow and Diffusion-based Models
Shokichi Takakura, Akifumi Wachi, Rei Higuchi +2
Inference-time alignment of flow and diffusion-based models is critical for achieving flexible generative modeling. Theoretically, Doob's -transform provides an elegant solution…
How Neural Reward Models Learn Features for Policy Optimization: A Single-Index Analysis
Rei Higuchi, Ryotaro Kawata, Akifumi Wachi +3
Reward modeling is not only a prediction problem: in KL-regularized policy optimization, the learned reward is exponentiated to define the deployed policy, so downstream value depe…
Wasserstein Saddle-Free Newton: Saddle Escape and Local Spectral Convergence
Razvan-Andrei Lascu, Taiji Suzuki
Extending saddle-free Newton methods to Wasserstein space requires controlling Hessian dependent operators along nonlinear transport trajectories and accounting for a Hessian spect…
Provably Learning Diffusion Models under the Manifold Hypothesis: Collapse and Refine
Wei Huang, Andi Han, Mingyuan Bai +4
Diffusion models generate high-dimensional data with remarkable quality, yet how their training efficiently learns the score function, bypassing the curse of dimensionality when da…
The Mechanism of Weak-to-Strong Generalization: Feature Elicitation from Latent Knowledge
Ryoya Awano, Taiji Suzuki
Weak-to-strong (W2S) generalization, in which a strong model is fine-tuned on outputs of a weaker, task-specialized model, has been proposed as an approach to aligning superhuman A…
Inference-Aware Meta-Alignment of LLMs via Non-Linear GRPO
Shokichi Takakura, Akifumi Wachi, Rei Higuchi +2
Aligning large language models (LLMs) to diverse human preferences is fundamentally challenging since criteria can often conflict with each other. Inference-time alignment methods…
A Relative-Budget Theory for Reinforcement Learning with Verifiable Rewards in Large Language Model Reasoning
Akifumi Wachi, Hirota Kinoshita, Shokichi Takakura +2
Reinforcement learning (RL) is a dominant paradigm for improving the reasoning abilities of large language models, yet its effectiveness varies across tasks and compute budgets. We…
Mean-field Analysis on Two-layer Neural Networks from a Kernel Perspective
Shokichi Takakura, Taiji Suzuki
In this paper, we study the feature learning ability of two-layer neural networks in the mean-field regime through the lens of kernel methods. To focus on the dynamics of the kerne…