3 papers
cs.AI2026
Small-Margin Preferences Still Matter-If You Train Them Right
Jinlong Pang, Zhaowei Zhu, Na Di +4
Preference optimization methods such as DPO align large language models (LLMs) using paired comparisons, but their effectiveness can be highly sensitive to the quality and difficul…
cs.CL2025
Reclaiming Low-Quality Instruction Data with Rule-Guided LLM Reconstruction
Zile Yang, Ling Li, Na Di +5
Data selection for supervised fine-tuning (SFT) often discards low-scoring instruction examples that may retain useful supervision. We propose a rule-guided LLM reconstruction fram…
cs.CL2025
Token Cleaning: Fine-Grained Data Selection for LLM Supervised Fine-Tuning
Jinlong Pang, Na Di, Zhaowei Zhu +4
Recent studies show that in supervised fine-tuning (SFT) of large language models (LLMs), data quality matters more than quantity. While most data cleaning methods concentrate on f…