6 papers
Muon with Nesterov Momentum: Heavy-Tailed Noise and (Randomized) Inexact Polar Decomposition
Sayantan Choudhury, Xiaoran Cheng, Martin TakÃ¡Ä +2
Most first-order optimizers treat matrix-valued parameters as vectors, ignoring the intrinsic geometry of hidden-layer weights in neural networks. Muon addresses this mismatch by u…
High Probability Complexity Bounds of Trust-Region Stochastic Sequential Quadratic Programming with Heavy-Tailed Noise
Yuchen Fang, Javad Lavaei, Sen Na
In this paper, we consider nonlinear optimization problems with a stochastic objective and deterministic equality constraints. We propose a Trust-Region Stochastic Sequential Quadr…
A Trust-Region Interior-Point Stochastic Sequential Quadratic Programming Method
Yuchen Fang, Jihun Kim, Sen Na +2
In this paper, we propose a trust-region interior-point stochastic sequential quadratic programming (TR-IP-SSQP) method for solving optimization problems with a stochastic objectiv…
Online Inference of Constrained Optimization: Primal-Dual Optimality and Sequential Quadratic Programming
Yihang Gao, Michael K. Ng, Michael W. Mahoney +1
We study online statistical inference for the solutions of stochastic optimization problems with equality and inequality constraints. Such problems are prevalent in statistics and…
Derivative-Free Sequential Quadratic Programming for Equality-Constrained Stochastic Optimization
Sen Na
We consider solving nonlinear optimization problems with a stochastic objective and deterministic equality constraints, assuming that only zero-order information is available for b…
Online Statistical Inference of Constrained Stochastic Optimization via Random Scaling
Xinchen Du, Wanrong Zhu, Wei Biao Wu +1
Constrained stochastic nonlinear optimization problems have attracted significant attention for their ability to model complex real-world scenarios in physics, economics, and biolo…