7 papers
CMuon: Accelerating and Stabilizing Diffusion Transformer Training via Chunked Momentum Orthogonalization
Chuyan Chen, Peng Sun, Kun Yuan
Diffusion Transformers (DiTs) have achieved state-of-the-art (SOTA) performance in visual generative modeling, yet their training remains computationally prohibitive. While the rec…
dRAE: Representation Autoencoder with Hyper-Spherical Codes
Tianren Ma, Lin Long, Chuyan Chen +4
In this work, we aim to discretize the high-dimensional visual representations to bridge the gap with language models - a non-trivial challenge, as existing quantization methods su…
Self-Adversarial One Step Generation via Condition Shifting
Deyuan Liu, Peng Sun, Yansen Han +3
The push for efficient text to image synthesis has moved the field toward one step sampling, yet existing methods still face a three way tradeoff among fidelity, inference speed, a…
An All-Reduce Compatible Top-K Compressor for Communication-Efficient Distributed Learning
Chuyan Chen, Chenyang Ma, Zhangxin Li +3
Communication remains a central bottleneck in large-scale distributed machine learning, and gradient sparsification has emerged as a promising strategy to alleviate this challenge.…
Greedy Low-Rank Gradient Compression for Distributed Learning with Convergence Guarantees
Chuyan Chen, Yutong He, Pengrui Li +2
Distributed optimization is pivotal for large-scale signal processing and machine learning, yet communication overhead remains a major bottleneck. Low-rank gradient compression, in…
From PowerSGD to PowerSGD+: Low-Rank Gradient Compression for Distributed Optimization with Convergence Guarantees
Shengping Xie, Chuyan Chen, Kun Yuan
Low-rank gradient compression methods, such as PowerSGD, have gained attention in communication-efficient distributed optimization. However, the convergence guarantees of PowerSGD…