5 papers · 1 filter
Muon with Nesterov Momentum: Heavy-Tailed Noise and (Randomized) Inexact Polar Decomposition
Sayantan Choudhury, Xiaoran Cheng, Martin TakÃ¡Ä +2
Most first-order optimizers treat matrix-valued parameters as vectors, ignoring the intrinsic geometry of hidden-layer weights in neural networks. Muon addresses this mismatch by u…
High Probability Complexity Bounds of Trust-Region Stochastic Sequential Quadratic Programming with Heavy-Tailed Noise
Yuchen Fang, Javad Lavaei, Sen Na
In this paper, we consider nonlinear optimization problems with a stochastic objective and deterministic equality constraints. We propose a Trust-Region Stochastic Sequential Quadr…
A Trust-Region Interior-Point Stochastic Sequential Quadratic Programming Method
Yuchen Fang, Jihun Kim, Sen Na +2
In this paper, we propose a trust-region interior-point stochastic sequential quadratic programming (TR-IP-SSQP) method for solving optimization problems with a stochastic objectiv…
Derivative-Free Sequential Quadratic Programming for Equality-Constrained Stochastic Optimization
Sen Na
We consider solving nonlinear optimization problems with a stochastic objective and deterministic equality constraints, assuming that only zero-order information is available for b…
Trust-Region Sequential Quadratic Programming for Stochastic Optimization with Random Models
Yuchen Fang, Sen Na, Michael W. Mahoney +1
In this work, we consider solving optimization problems with a stochastic objective and deterministic equality constraints. We propose a Trust-Region Sequential Quadratic Programmi…