2 papers
cs.CL2025
Speculative Knowledge Distillation: Bridging the Teacher-Student Gap Through Interleaved Sampling
Wenda Xu, Rujun Han, Zifeng Wang +7
Recent advances in knowledge distillation (KD) have enabled smaller student models to approach the performance of larger teacher models. However, popular methods such as supervised…
cs.LG2024
BPO: Staying Close to the Behavior LLM Creates Better Online LLM Alignment
Wenda Xu, Jiachen Li, William Yang Wang +1
Direct alignment from preferences (DAP) has emerged as a promising paradigm for aligning large language models (LLMs) to human desiderata from pre-collected, offline preference dat…