2 citations · 2 across the 9 of their papers we have counts for
4 papers · 1 filter
Memory-Efficient 4-bit Preconditioned Stochastic Optimization
Jingyang Li, Kuangyu Ding, Kim-Chuan Toh +1
Preconditioned stochastic optimization algorithms, exemplified by Shampoo, outperform first-order optimizers by offering theoretical convergence benefits and practical gains in lar…
Optimization Hyper-parameter Laws for Large Language Models
Xingyu Xie, Kuangyu Ding, Shuicheng Yan +2
Large Language Models have driven significant AI advancements, yet their training is resource-intensive and highly sensitive to hyper-parameter selection. While scaling laws provid…
Developing Lagrangian-based Methods for Nonsmooth Nonconvex Optimization
Nachuan Xiao, Kuangyu Ding, Xiaoyin Hu +1
In this paper, we consider the minimization of a nonsmooth nonconvex objective function over a closed convex subset of , with additional nonsmoot…
Stochastic Bregman Subgradient Methods for Nonsmooth Nonconvex Optimization Problems
Kuangyu Ding, Kim-Chuan Toh
This paper focuses on the problem of minimizing a locally Lipschitz continuous function. Motivated by the effectiveness of Bregman gradient methods in training nonsmooth deep neura…