7 papers
Aligning Language Models for Lyric-to-Melody Generation with Rule-Based Musical Constraints
Hao Meng, Siyuan Zheng, Shuran Zhou +2
Large Language Models (LLMs) show promise in lyric-to-melody generation, but models trained with Supervised Fine-Tuning (SFT) often produce musically implausible melodies with issu…
Multi-Turn Interactions for Text-to-SQL with Large Language Models
Guanming Xiong, Junwei Bao, Hongfei Jiang +2
This study explores text-to-SQL parsing by leveraging the powerful reasoning capabilities of large language models (LLMs). Despite recent advancements, existing LLM-based methods a…
GVPO: Group Variance Policy Optimization for Large Language Model Post-Training
Kaichen Zhang, Yuzhong Hong, Junwei Bao +4
Post-training plays a crucial role in refining and aligning large language models to meet specific tasks and human preferences. While recent advancements in post-training technique…
RSPO: Risk-Seeking Policy Optimization for Pass@k and Max@k Metrics in Large Language Models
Kaichen Zhang, Shenghao Gao, Yuzhong Hong +6
Current large language model post-training optimizes a risk-neutral objective that maximizes expected reward, yet evaluation relies heavily on risk-seeking metrics like Pass@k (at…
Energy-Based Preference Model Offers Better Offline Alignment than the Bradley-Terry Preference Model
Yuzhong Hong, Hanshan Zhang, Junwei Bao +2
Since the debut of DPO, it has been shown that aligning a target LLM with human preferences via the KL-constrained RLHF loss is mathematically equivalent to a special kind of rewar…
Preference-Oriented Supervised Fine-Tuning: Favoring Target Model Over Aligned Large Language Models
Yuchen Fan, Yuzhong Hong, Qiushi Wang +3
Alignment, endowing a pre-trained Large language model (LLM) with the ability to follow instructions, is crucial for its real-world applications. Conventional supervised fine-tunin…