1 citations · 1 across the 8 of their papers we have counts for
4 papers · 1 filter
Normalized Rewards for Preference Optimization
Shawn Im, Federico Danieli, Skyler Seto +2
Direct Alignment Algorithms (DAAs) such as DPO have become a common way to post-train and align LLMs with human preferences. However, DAAs have been observed to over-optimize their…
Locking Pretrained Weights via Deep Low-Rank Residual Distillation
Keitaro Sakamoto, Pierre Ablin, Federico Danieli +1
The quality of open-weight language models has dramatically improved in recent years. Sharing weights greatly facilitates model adoption by enabling their use across diverse hardwa…
Theoretical Limits of Language Model Alignment
Lucas Monteiro Paes, Natalie Mackraz, Barry-John Theobald +1
Language model (LM) alignment improves model outputs to reflect human preferences while preserving the capabilities of the base model. The most common alignment approaches are (i)…
GenCtrl -- A Formal Controllability Toolkit for Generative Models
Emily Cheng, Carmen Amo Alonso, Federico Danieli +4
As generative models become ubiquitous, there is a critical need for fine-grained control over the generation process. Yet, while controlled generation methods from prompting to fi…