4 papers
Hard Labels In! Rethinking the Role of Hard Labels in Mitigating Local Semantic Drift
Jiacheng Cui, Bingkui Tong, Xinyue Bi +3
Soft labels from teacher models are a de facto practice for knowledge transfer and large-scale dataset distillation (e.g., SRe2L, LPLD). However, when we limit the number of crops…
LLMSurgeon: Diagnosing Data Mixture of Large Language Models
Yaxin Luo, Jiacheng Cui, Xiaohan Zhao +5
The pretraining data mixture of Large Language Models (LLMs) constitutes their "digital DNA", shaping model behaviors, capabilities, and failure modes. Yet this composition is rare…
Dataset Distillation via Committee Voting
Jiacheng Cui, Zhaoyi Li, Xiaochen Ma +3
Dataset distillation aims to synthesize a compact yet representative dataset that preserves the essential characteristics of the original data for efficient model training. Existin…
FADRM: Fast and Accurate Data Residual Matching for Dataset Distillation
Jiacheng Cui, Xinyue Bi, Yaxin Luo +3
Residual connection has been extensively studied and widely applied at the model architecture level. However, its potential in the more challenging data-centric approaches remains…