5 papers
Policy Gradient Algorithms in Average-Reward Multichain MDPs
Jongmin Lee, Ernest K. Ryu
While there is an extensive body of research analyzing policy gradient methods for discounted cumulative-reward MDPs, prior work on policy gradient methods for average-reward MDPs…
STORK: Faster Diffusion And Flow Matching Sampling By Resolving Both Stiffness And Structure-Dependence
Zheng Tan, Weizhen Wang, Andrea L. Bertozzi +1
Diffusion models (DMs) and flow-matching models have demonstrated remarkable performance in image and video generation. However, such models require a significant number of functio…
Numerical Analysis of HiPPO-LegS ODE for Deep State Space Models
Jaesung R. Park, Jaewook J. Suh, Youngjoon Hong +1
In deep learning, the recently introduced state space models utilize HiPPO (High-order Polynomial Projection Operators) memory units to approximate continuous-time trajectories of…
LoRA Training Provably Converges to a Low-Rank Global Minimum or It Fails Loudly (But it Probably Won't Fail)
Junsu Kim, Jaeyeon Kim, Ernest K. Ryu
Low-rank adaptation (LoRA) has become a standard approach for fine-tuning large foundation models. However, our theoretical understanding of LoRA remains limited as prior analyses…
Optimization Algorithm Design via Electric Circuits
Stephen P. Boyd, Tetiana Parshakova, Ernest K. Ryu +1
We present a novel methodology for convex optimization algorithm design using ideas from electric RLC circuits. Given an optimization problem, the first stage of the methodology is…