10 papers
An Adaptive, Parallel, and Inexact Newton Method for Large-scale Nonlinear Optimal Control
Luke Bhan, Michael W. Mahoney, Sen Na
We design adaptive overlapping temporal decomposition (AOTD), a parallel sequential quadratic programming (SQP) method for long-horizon nonlinear optimal control problems (OCPs). A…
TR-SSQP: A Trust-Region Method for Constrained Stochastic Optimization under Heavy-Tailed Noise
Haoxuan Wang, Yuchen Fang, Sen Na
We consider stochastic nonlinear optimization problems with deterministic equality constraints. While unconstrained stochastic optimization is well understood, the interplay betwee…
Inference for Newton Methods with Accelerated Sketch-and-Project via Random Scaling
Xinchen Du, Elizaveta Rebrova, Michał Dereziński +1
We study an online sketched Newton method that approximates the Newton direction at each step via a state-of-the-art sketching solver, called the generalized accelerated sketch-and…
Muon with Nesterov Momentum: Heavy-Tailed Noise and (Randomized) Inexact Polar Decomposition
Sayantan Choudhury, Xiaoran Cheng, Martin Takáč +2
Most first-order optimizers treat matrix-valued parameters as vectors, ignoring the intrinsic geometry of hidden-layer weights in neural networks. Muon addresses this mismatch by u…
A Trust-Region Interior-Point Stochastic Sequential Quadratic Programming Method
Yuchen Fang, Jihun Kim, Sen Na +2
In this paper, we propose a trust-region interior-point stochastic sequential quadratic programming (TR-IP-SSQP) method for solving optimization problems with a stochastic objectiv…
Online Inference of Constrained Optimization: Primal-Dual Optimality and Sequential Quadratic Programming
Yihang Gao, Michael K. Ng, Michael W. Mahoney +1
We study online statistical inference for the solutions of stochastic optimization problems with equality and inequality constraints. Such problems are prevalent in statistics and…
Derivative-Free Sequential Quadratic Programming for Equality-Constrained Stochastic Optimization
Sen Na
We consider solving nonlinear optimization problems with a stochastic objective and deterministic equality constraints, assuming that only zero-order information is available for b…
Online Statistical Inference of Constrained Stochastic Optimization via Random Scaling
Xinchen Du, Wanrong Zhu, Wei Biao Wu +1
Constrained stochastic nonlinear optimization problems have attracted significant attention for their ability to model complex real-world scenarios in physics, economics, and biolo…
High Probability Complexity Bounds of Trust-Region Stochastic Sequential Quadratic Programming with Heavy-Tailed Noise
Yuchen Fang, Javad Lavaei, Sen Na
In this paper, we consider nonlinear optimization problems with a stochastic objective and deterministic equality constraints. We propose a Trust-Region Stochastic Sequential Quadr…
Trust-Region Sequential Quadratic Programming for Stochastic Optimization with Random Models
Yuchen Fang, Sen Na, Michael W. Mahoney +1
In this work, we consider solving optimization problems with a stochastic objective and deterministic equality constraints. We propose a Trust-Region Sequential Quadratic Programmi…