1 citations · 2 across the 4 of their papers we have counts for
3 papers · 1 filter
Memory-Efficient 4-bit Preconditioned Stochastic Optimization
Jingyang Li, Kuangyu Ding, Kim-Chuan Toh +1
Preconditioned stochastic optimization algorithms, exemplified by Shampoo, outperform first-order optimizers by offering theoretical convergence benefits and practical gains in lar…
Towards Understanding Why FixMatch Generalizes Better Than Supervised Learning
Jingyang Li, Jiachun Pan, Vincent Y. F. Tan +2
Semi-supervised learning (SSL), exemplified by FixMatch (Sohn et al., 2020), has shown significant generalization advantages over supervised learning (SL), particularly in the cont…
LoCo: Low-Bit Communication Adaptor for Large-scale Model Training
Xingyu Xie, Zhijie Lin, Kim-Chuan Toh +1
To efficiently train large-scale models, low-bit gradient communication compresses full-precision gradients on local GPU nodes into low-precision ones for higher gradient synchroni…