2 citations · 2 across the 9 of their papers we have counts for
11 papers
From Mixing to Tearing: Graph Decomposition in Decentralized Optimization via Message Passing
Kuangyu Ding, Gesualdo Scutari
We study decentralized minimization of sums of smooth strongly convex functions, one per agent, with communication restricted to neighbors in a graph. Existing decentralized algori…
Establishing Boundary KKT Convergence of Mirror Descent through Reparameterization
Kuangyu Ding, Kim-Chuan Toh
Sequence convergence to a boundary Karush--Kuhn--Tucker (KKT) point has long remained unclear for nonconvex mirror descent with Legendre kernels. The difficulty arises from the blo…
Non-KKT Accumulation in Entropic Mirror Descent
Kuangyu Ding, Kim-Chuan Toh
For mirror descent generated by a Legendre kernel, perhaps one of the most basic question in optimization is this: must every accumulation point of a bounded mirror descent sequenc…
A New Decomposition Paradigm for Graph-structured Nonlinear Programs via Message Passing
Kuangyu Ding, Marie Maros, Gesualdo Scutari
We study finite-sum nonlinear programs with localized variable coupling encoded by a (hyper)graph. We introduce a graph-compliant decomposition framework that brings message passin…
On exploration of an interior mirror descent flow for stochastic nonconvex constrained problem
Kuangyu Ding, Kim-Chuan Toh
We study a nonsmooth nonconvex optimization problem defined over nonconvex constraints, where the feasible set is given by the intersection of the closure of an open set and a smoo…
Memory-Efficient 4-bit Preconditioned Stochastic Optimization
Jingyang Li, Kuangyu Ding, Kim-Chuan Toh +1
Preconditioned stochastic optimization algorithms, exemplified by Shampoo, outperform first-order optimizers by offering theoretical convergence benefits and practical gains in lar…
Optimization Hyper-parameter Laws for Large Language Models
Xingyu Xie, Kuangyu Ding, Shuicheng Yan +2
Large Language Models have driven significant AI advancements, yet their training is resource-intensive and highly sensitive to hyper-parameter selection. While scaling laws provid…
Developing Lagrangian-based Methods for Nonsmooth Nonconvex Optimization
Nachuan Xiao, Kuangyu Ding, Xiaoyin Hu +1
In this paper, we consider the minimization of a nonsmooth nonconvex objective function over a closed convex subset of , with additional nonsmoot…
Stochastic Bregman Subgradient Methods for Nonsmooth Nonconvex Optimization Problems
Kuangyu Ding, Kim-Chuan Toh
This paper focuses on the problem of minimizing a locally Lipschitz continuous function. Motivated by the effectiveness of Bregman gradient methods in training nonsmooth deep neura…
Adam-family Methods with Decoupled Weight Decay in Deep Learning
Kuangyu Ding, Nachuan Xiao, Kim-Chuan Toh
In this paper, we investigate the convergence properties of a wide class of Adam-family methods for minimizing quadratically regularized nonsmooth nonconvex optimization problems,…
On proximal augmented Lagrangian based decomposition methods for dual block-angular convex composite programming problems
Kuang-Yu Ding, Xin-Yee Lam, Kim-Chuan Toh
We design inexact proximal augmented Lagrangian based decomposition methods for convex composite programming problems with dual block-angular structures. Our methods are particular…