10 citations · 14 across the 15 of their papers we have counts for
3 papers · 1 filter
Rubric-Based Optimization for Text-to-Music Generation
Ping Wang, Guang Yang, Shao-Rong Su +3
Post-training text-to-music generation requires reward signals that capture multiple aspects of musical quality beyond what any single automatic metric can measure. We study struct…
Knowledge Distillation During Mid-Training Favors Reasoning over Factual Recall
Jacqueline He, Howard Yen, Shuyue Stella Li +9
Logit-based knowledge distillation (KD) is used to train smaller language models (LMs) via supervision from stronger teachers, but whether its benefits are consistent across traini…
Spokes: Optimizing for Diverse Pretraining Data Selection
Clarence Lee, Yejin Choi, Luke Zettlemoyer +2
Diversity plays a critical role in data selection, improving performance under fixed data budgets by reducing redundancy and repetition. However, optimizing for diversity is inhere…