Showing 2024Show all
3 papers · 1 filter
cs.LG2024
Energy-Based Preference Model Offers Better Offline Alignment than the Bradley-Terry Preference Model
Yuzhong Hong, Hanshan Zhang, Junwei Bao +2
Since the debut of DPO, it has been shown that aligning a target LLM with human preferences via the KL-constrained RLHF loss is mathematically equivalent to a special kind of rewar…
cs.CL2024
Preference-Oriented Supervised Fine-Tuning: Favoring Target Model Over Aligned Large Language Models
Yuchen Fan, Yuzhong Hong, Qiushi Wang +3
Alignment, endowing a pre-trained Large language model (LLM) with the ability to follow instructions, is crucial for its real-world applications. Conventional supervised fine-tunin…
cs.CL2024
BoRA: Bi-dimensional Weight-Decomposed Low-Rank Adaptation
Qiushi Wang, Yuchen Fan, Junwei Bao +2
In recent years, Parameter-Efficient Fine-Tuning (PEFT) methods like Low-Rank Adaptation (LoRA) have significantly enhanced the adaptability of large-scale pre-trained models. Weig…