565 citations · 1k across the 23 of their papers we have counts for
4 papers · 2 filters
Compositional preference models for aligning LMs
Dongyoung Go, Tomasz Korbak, Germán Kruszewski +2
As language models (LMs) become more capable, it is increasingly important to align them with human preferences. However, the dominant paradigm for training Preference Models (PMs)…
Should you marginalize over possible tokenizations?
Nadezhda Chirkova, Germán Kruszewski, Jos Rozen +1
Autoregressive language models (LMs) map token sequences to probabilities. The usual practice for computing the probability of any character string (e.g. English sentences) is to f…
disco: a toolkit for Distributional Control of Generative Models
Germán Kruszewski, Jos Rozen, Marc Dymetman
Pre-trained language models and other generative models have revolutionized NLP and beyond. However, these models tend to reproduce undesirable biases present in their training dat…
Aligning Language Models with Preferences through f-divergence Minimization
Dongyoung Go, Tomasz Korbak, Germán Kruszewski +3
Aligning language models with preferences can be posed as approximating a target distribution representing some desired behavior. Existing approaches differ both in the functional…