1 citations · 2 across the 4 of their papers we have counts for
4 papers
Preference Optimization via Contrastive Divergence: Your Reward Model is Secretly an NLL Estimator
Zhuotong Chen, Fang Liu, Xuan Zhu +2
Existing studies on preference optimization (PO) have centered on constructing pairwise preference data following simple heuristics, such as maximizing the margin between preferred…
AIDE: Attribute-Guided MultI-Hop Data Expansion for Data Scarcity in Task-Specific Fine-tuning
Jiayu Li, Xuan Zhu, Fang Liu +1
Fine-tuning large language models (LLMs) for specific tasks requires diverse, high-quality training data. However, obtaining sufficient relevant data remains a significant challeng…
Towards Improved Preference Optimization Pipeline: from Data Generation to Budget-Controlled Regularization
Zhuotong Chen, Fang Liu, Jennifer Zhu +2
Direct Preference Optimization (DPO) and its variants have become the de facto standards for aligning large language models (LLMs) with human preferences or specific goals. However…
DFlow: Diverse Dialogue Flow Simulation with Large Language Models
Wanyu Du, Song Feng, James Gung +4
Developing language model-based dialogue agents requires effective data to train models that can follow specific task logic. However, most existing data simulation methods focus on…