Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
Latent-MOPD: Latent Multi-Teacher On-Policy Distillation
Zhengyu Fang, Seoyeon Hong, Jie Yang +4
On-policy distillation (OPD) trains a student on the responses it generates. Existing LLM multi-teacher OPD transfers what specialists predict through their output distributions. W…
cs.LG2026
D-QRELO: Training- and Data-Free Delta Compression for Large Language Models via Quantization and Residual Low-Rank Approximation
Junlin Li, Shuangyong Song, Guodong Du +6
Supervised Fine-Tuning (SFT) accelerates taskspecific large language models (LLMs) development, but the resulting proliferation of finetuned models incurs substantial memory overhe…
cs.LG2026
Dataset-Level Metrics Attenuate Non-Determinism: A Fine-Grained Non-Determinism Evaluation in Diffusion Language Models
Zhengyu Fang, Zhimeng Jiang, Huiyuan Chen +5
Diffusion language models (DLMs) have emerged as a promising paradigm for large language models (LLMs), yet the non-deterministic behavior of DLMs remains poorly understood. The ex…