8 papers
CODEBLOCK: Learning to Supervise Code at the Right Granularity
Zhijie Deng, Ling Li, Jinlong Pang +4
Supervised fine-tuning of code LLMs typically applies uniform cross-entropy loss to all response tokens, implicitly assuming that every token provides equally useful learning signa…
Automatic Dataset Construction (ADC): Sample Collection, Data Curation, and Beyond
Minghao Liu, Zonglin Di, Jiaheng Wei +15
Large-scale data collection is essential for developing personalized training data, mitigating the shortage of training data, and fine-tuning specialized models. However, creating…
Token Cleaning: Fine-Grained Data Selection for LLM Supervised Fine-Tuning
Jinlong Pang, Na Di, Zhaowei Zhu +4
Recent studies show that in supervised fine-tuning (SFT) of large language models (LLMs), data quality matters more than quantity. While most data cleaning methods concentrate on f…
Small-Margin Preferences Still Matter-If You Train Them Right
Jinlong Pang, Zhaowei Zhu, Na Di +4
Preference optimization methods such as DPO align large language models (LLMs) using paired comparisons, but their effectiveness can be highly sensitive to the quality and difficul…
Evaluating LLM-Contaminated Crowdsourcing Data Without Ground Truth
Yichi Zhang, Jinlong Pang, Zhaowei Zhu +1
The recent success of generative AI highlights the crucial role of high-quality human feedback in building trustworthy AI systems. However, the increasing use of large language mod…
ENTP: Enhancing Low-Quality SFT Data via Neural-Symbolic Text Purge-Mix
Zile Yang, Ling Li, Na Di +5
Supervised Fine-Tuning (SFT) adapts pre-trained Large Language Models (LLMs) to domain-specific instructions by training on a carefully curated subset of high-quality instruction-r…