7 papers
RLTLDR: Self-improvement by Internalizing Self-generated Feedback
Michael Kirchhof, Eleonora Gualdoni, Andrew Szot +6
The common paradigm of reinforcement learning with verifiable rewards (RLVR) is to let agents make multiple attempts at a task, and optimize towards the successful ones. This becom…
Cartridges++: KV Cache Compression without Off-Context Derailment
Sonia Laguna, Joao Monteiro, Marco Cuturi +2
Serving long documents to a Large Language Model (LLM) repeatedly is expensive: computations grow with context length, and the memory footprint of the key-value (KV) cache balloons…
DynaMiCS: Fine-tuning LLMs with Performance Constraints using Dynamic Mixtures
Eleonora Gualdoni, Sonia Laguna, Louis Bethune +3
Multi-domain fine-tuning of large language models requires improving performance on target domains while preserving previous capabilities, such as general knowledge, instruction fo…
HyperTransport: Amortized Conditioning of T2I Generative Models
Valentino Maiorca, Eleonora Gualdoni, Xavier Suau +3
As foundation models grow in capability, the ability to efficiently and reliably control their behavior becomes critical. Fine-tuning these models can be costly, and while promptin…
What do your logits know?
Masha Fedzechkina, Eleonora Gualdoni, Rita Ramos +1
Recent work has shown that probing model internals can reveal a wealth of information not apparent from the model generations. This poses a risk of unintentional or malicious infor…
LinEAS: End-to-end Learning of Activation Steering with a Distributional Loss
Pau Rodriguez, Michal Klein, Eleonora Gualdoni +5
The growing use of generative models in daily life calls for efficient mechanisms to control their generation, to e.g., produce safe content or provide users with tools to explore…
Scaling Laws for Forgetting during Finetuning with Pretraining Data Injection
Louis Bethune, David Grangier, Dan Busbridge +3
A widespread strategy to obtain a language model that performs well on a target domain is to finetune a pretrained model to perform unsupervised next-token prediction on data from…