3 papers
cs.LG2026
Small LLMs: Pruning vs. Training from Scratch
Yufeng Xu, Taiming Lu, Kunjun Li +3
Pruning promises a shortcut to strong small language models. In this work, we examine this promise by pruning Llama-3.1-8B at pruning ratios of 0.5--0.8 with six methods spanning d…
cs.LG2026
Multi-Token Residual Prediction
Yufeng Xu, Zishuo Bao, Qian Wang +6
Diffusion Language Models (DLMs) generate text by iteratively denoising masked token sequences, offering a tradeoff between parallelism and quality compared to autoregressive model…
cs.LG2026
Good SFT Optimizes for SFT, Better SFT Prepares for Reinforcement Learning
Dylan Zhang, Yufeng Xu, Haojin Wang +2
Post-training of reasoning LLMs is a holistic process that typically consists of an offline SFT stage followed by an online reinforcement learning (RL) stage. However, SFT is often…