7 papers
Establishing Boundary KKT Convergence of Mirror Descent through Reparameterization
Kuangyu Ding, Kim-Chuan Toh
Sequence convergence to a boundary Karush--Kuhn--Tucker (KKT) point has long remained unclear for nonconvex mirror descent with Legendre kernels. The difficulty arises from the blo…
Non-KKT Accumulation in Entropic Mirror Descent
Kuangyu Ding, Kim-Chuan Toh
For mirror descent generated by a Legendre kernel, perhaps one of the most basic question in optimization is this: must every accumulation point of a bounded mirror descent sequenc…
Optimization Hyper-parameter Laws for Large Language Models
Xingyu Xie, Kuangyu Ding, Shuicheng Yan +2
Large Language Models have driven significant AI advancements, yet their training is resource-intensive and highly sensitive to hyper-parameter selection. While scaling laws provid…
A New Decomposition Paradigm for Graph-structured Nonlinear Programs via Message Passing
Kuangyu Ding, Marie Maros, Gesualdo Scutari
We study finite-sum nonlinear programs with localized variable coupling encoded by a (hyper)graph. We introduce a graph-compliant decomposition framework that brings message passin…
On exploration of an interior mirror descent flow for stochastic nonconvex constrained problem
Kuangyu Ding, Kim-Chuan Toh
We study a nonsmooth nonconvex optimization problem defined over nonconvex constraints, where the feasible set is given by the intersection of the closure of an open set and a smoo…
Stochastic Bregman Subgradient Methods for Nonsmooth Nonconvex Optimization Problems
Kuangyu Ding, Kim-Chuan Toh
This paper focuses on the problem of minimizing a locally Lipschitz continuous function. Motivated by the effectiveness of Bregman gradient methods in training nonsmooth deep neura…