16 papers
Shieldstral
Antonia Calvi, Avinash Sooriyarachchi, Giada Pistilli +274
We introduce Shieldstral, a 3B-parameter policy-adaptive multimodal safety classifier that matches or outperforms models nearly 7 its size on text safety benchmarks and set…
Simplifying the Modeling of Arbitrary Conditionals in Natural Language
Yinhan Lu, Eric Elmoznino, Léo Gagnon +3
Causal Transformers model sequences through an autoregressive factorization of the joint distribution, which enables efficient left-to-right decoding and conditional likelihood com…
Reinforced sequential Monte Carlo for amortised sampling
Sanghyeok Choi, Sarthak Mittal, VÃctor Elvira +2
This paper proposes a synergy of amortised and particle-based methods for sampling from distributions defined by unnormalised density functions. We state a connection between seque…
Beyond Distribution Sharpening: The Importance of Task Rewards
Sarthak Mittal, Leo Gagnon, Guillaume Lajoie
Frontier models have demonstrated exceptional capabilities following the integration of task-reward-based reinforcement learning (RL) into their training pipelines, enabling system…
A Compression Perspective on Simplicity Bias
Tom Marty, Eric Elmoznino, Leo Gagnon +5
Deep neural networks exhibit a simplicity bias, a well-documented tendency to favor simple functions over complex ones. In this work, we cast new light on this phenomenon through t…
A Comedy of Estimators: On KL Regularization in RL Training of LLMs
Vedant Shah, Johan Obando-Ceron, Vineet Jain +10
The reasoning performance of large language models (LLMs) can be substantially improved by training them with reinforcement learning (RL). The RL objective for LLM training involve…