8 citations · 10 across the 10 of their papers we have counts for
11 papers
No Pain, More Gain: Iterative Merging for Effective Multi-Teacher On-Policy Distillation
Seonghyeon Kim, Chaeyun Jang, Noah Lee +2
Multi-teacher on-policy distillation (MOPD) combines independently developed domain teachers into a single student by distilling their predictions on student-generated samples. We…
Persona-Pruner: Sculpting Lightweight Models for Role-Playing
Jinsu Kim, Jihoon Tack, Noah Lee +1
Language Models (LMs) have shown remarkable potential as role-playing chatbots, delivering consistent, stylized interactions when given a specification of a character or user perso…
On the Robustness of Reward Models for Language Model Alignment
Jiwoo Hong, Noah Lee, Eunki Kim +5
The Bradley-Terry (BT) model is widely practiced in reward modeling for reinforcement learning with human feedback (RLHF). Despite its effectiveness, reward models (RMs) trained wi…
AlphaPO: Reward Shape Matters for LLM Alignment
Aman Gupta, Shao Tang, Qingquan Song +10
Reinforcement Learning with Human Feedback (RLHF) and its variants have made huge strides toward the effective alignment of large language models (LLMs) to follow instructions and…
Evaluating the Consistency of LLM Evaluators
Noah Lee, Jiwoo Hong, James Thorne
Large language models (LLMs) have shown potential as general evaluators along with the evident benefits of speed and cost. While their correlation against human annotators has been…
Cross-lingual Transfer of Reward Models in Multilingual Alignment
Jiwoo Hong, Noah Lee, Rodrigo Martínez-Castaño +2
Reinforcement learning with human feedback (RLHF) is shown to largely benefit from precise reward models (RMs). However, recent studies in reward modeling schemes are skewed toward…
Margin-aware Preference Optimization for Aligning Diffusion Models without Reference
Jiwoo Hong, Sayak Paul, Noah Lee +3
Modern preference alignment methods, such as DPO, rely on divergence regularization to a reference model for training stability-but this creates a fundamental problem we call "refe…
The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Models
Seungone Kim, Juyoung Suk, Ji Yong Cho +29
As language models (LMs) become capable of handling a wide range of tasks, their evaluation is becoming as challenging as their development. Most generation benchmarks currently as…
ORPO: Monolithic Preference Optimization without Reference Model
Jiwoo Hong, Noah Lee, James Thorne
While recent preference alignment algorithms for language models have demonstrated promising results, supervised fine-tuning (SFT) remains imperative for achieving successful conve…
Redrawing the 2012 map of the Maryland congressional districts
Noah Lee, Hyunwoo Park, Sangho Shim
Gerrymandering is the practice of drawing biased electoral maps that manipulate the voter population to gain an advantage. The most recent time gerrymandering became an issue was 2…
Robust Fine-Tuning of Vision-Language Models for Domain Generalization
Kevin Vogt-Lowell, Noah Lee, Theodoros Tsiligkaridis +1
Transfer learning enables the sharing of common knowledge among models for a variety of downstream tasks, but traditional methods suffer in limited training data settings and produ…