1 citations · 2 across the 7 of their papers we have counts for
7 papers
Towards On-Policy SFT: Distribution Discriminant Theory and its Applications in LLM Training
Miaosen Zhang, Yishan Liu, Shuxia Lin +8
Supervised fine-tuning (SFT) is computationally efficient but often yields inferior generalization compared to reinforcement learning (RL). This gap is primarily driven by RL's use…
CONCORD: Concept-Informed Diffusion for Dataset Distillation
Jianyang Gu, Haonan Wang, Ruoxi Jia +4
Dataset distillation (DD) has witnessed significant progress in creating small datasets that encapsulate rich information from large original ones. Particularly, methods based on g…
Improved Methods for Model Pruning and Knowledge Distillation
Wei Jiang, Anying Fu, Youling Zhang
Model pruning is a performance optimization technique for large language models like R1 or o3-mini. However, existing pruning methods often lead to significant performance degradat…
ChorusCVR: Chorus Supervision for Entire Space Post-Click Conversion Rate Modeling
Wei Cheng, Yucheng Lu, Boyang Xia +9
Post-click conversion rate (CVR) estimation is a vital task in many recommender systems of revenue businesses, e.g., e-commerce and advertising. In a perspective of sample, a typic…
Group Distributionally Robust Dataset Distillation with Risk Minimization
Saeed Vahidian, Mingyu Wang, Jianyang Gu +3
Dataset distillation (DD) has emerged as a widely adopted technique for crafting a synthetic dataset that captures the essential information of a training dataset, facilitating the…
Efficient Dataset Distillation via Minimax Diffusion
Jianyang Gu, Saeed Vahidian, Vyacheslav Kungurtsev +4
Dataset distillation reduces the storage and computational consumption of training a network by generating a small surrogate dataset that encapsulates rich information of the origi…
DREAM+: Efficient Dataset Distillation by Bidirectional Representative Matching
Yanqing Liu, Jianyang Gu, Kai Wang +4
Dataset distillation plays a crucial role in creating compact datasets with similar training performance compared with original large-scale ones. This is essential for addressing t…