5 papers · 1 filter
SALAAD: Sparse And Low-Rank Adaptation via ADMM for Large Language Model Inference
Hao Ma, Melis Ilayda Bal, Liang Zhang +4
Modern large language models are increasingly deployed under compute and memory constraints, making flexible control of model capacity a central challenge. While sparse and low-ran…
Beyond Reward Hacking: Causal Rewards for Large Language Model Alignment
Chaoqi Wang, Zhuokai Zhao, Yibo Jiang +8
Recent advances in large language models (LLMs) have demonstrated significant progress in performing complex tasks. While Reinforcement Learning from Human Feedback (RLHF) has been…
Preference Optimization with Multi-Sample Comparisons
Chaoqi Wang, Zhuokai Zhao, Chen Zhu +8
Recent advancements in generative models, particularly large language models (LLMs) and diffusion models, have been driven by extensive pretraining on large datasets followed by po…
Step-KTO: Optimizing Mathematical Reasoning through Stepwise Binary Feedback
Yen-Ting Lin, Di Jin, Tengyu Xu +11
Large language models (LLMs) have recently demonstrated remarkable success in mathematical reasoning. Despite progress in methods like chain-of-thought prompting and self-consisten…
The Perfect Blend: Redefining RLHF with Mixture of Judges
Tengyu Xu, Eryk Helenowski, Karthik Abinav Sankararaman +17
Reinforcement learning from human feedback (RLHF) has become the leading approach for fine-tuning large language models (LLM). However, RLHF has limitations in multi-task learning…