1 citations · 1 across the 13 of their papers we have counts for
Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
DemoPSD: Disagreement-Modulated Policy Self-Distillation
Yunhe Li, Hao Shi, Wenhao Liu +5
On-policy self-distillation (OPSD) has emerged as a practical method for training large language models (LLMs) to reason, where a single model acts as both the teacher and the stud…
cs.LG2024
Improved Quantization Strategies for Managing Heavy-tailed Gradients in Distributed Learning
Guangfeng Yan, Tan Li, Yuanzhang Xiao +2
Gradient compression has surfaced as a key technique to address the challenge of communication efficiency in distributed learning. In distributed deep learning, however, it is obse…
cs.LG2024
Truncated Non-Uniform Quantization for Distributed SGD
Guangfeng Yan, Tan Li, Yuanzhang Xiao +2
To address the communication bottleneck challenge in distributed learning, our work introduces a novel two-stage quantization strategy designed to enhance the communication efficie…