7 papers
Self-Rewarding PPO: Aligning Large Language Models with Demonstrations Only
Qingru Zhang, Liang Qiu, Ilgee Hong +11
Supervised fine-tuning (SFT) has emerged as a crucial method for aligning large language models (LLMs) with human-annotated demonstrations. However, SFT, being an off-policy approa…
POPI: Personalizing LLMs via Optimized Natural Language Preference Inference
Yizhuo Chen, Xin Liu, Ruijie Wang +7
Large language models (LLMs) are typically aligned with population-level preferences, despite substantial variation across individual users. We introduce POPI, a user-level persona…
Instant Personalized Large Language Model Adaptation via Hypernetwork
Zhaoxuan Tan, Zixuan Zhang, Haoyang Wen +8
Personalized large language models (LLMs) tailor content to individual preferences using user profiles or histories. However, existing parameter-efficient fine-tuning (PEFT) method…
Aligning Large Language Models with Implicit Preferences from User-Generated Content
Zhaoxuan Tan, Zheng Li, Tianyi Liu +10
Learning from preference feedback is essential for aligning large language models (LLMs) with human values and improving the quality of generated responses. However, existing prefe…
IHEval: Evaluating Language Models on Following the Instruction Hierarchy
Zhihan Zhang, Shiyang Li, Zixuan Zhang +11
The instruction hierarchy, which establishes a priority order from system messages to user messages, conversation history, and tool outputs, is essential for ensuring consistent an…
Personalized News Recommendation System via LLM Embedding and Co-Occurrence Patterns
Zheng Li, Kai Zhange
In the past two years, large language models (LLMs) have achieved rapid development and demonstrated remarkable emerging capabilities. Concurrently, with powerful semantic understa…
Evolutionary Contrastive Distillation for Language Model Alignment
Julian Katz-Samuels, Zheng Li, Hyokun Yun +5
The ability of large language models (LLMs) to execute complex instructions is essential for their real-world applications. However, several recent studies indicate that LLMs strug…