34 citations · 45 across the 13 of their papers we have counts for
4 papers · 1 filter
Exploring Quantization for Efficient Pre-Training of Transformer Language Models
Kamran Chitsaz, Quentin Fournier, Gonçalo Mordido +1
The increasing scale of Transformer models has led to an increase in their pre-training computational requirements. While quantization has proven to be effective after pre-training…
Why Don't Prompt-Based Fairness Metrics Correlate?
Abdelrahman Zayed, Goncalo Mordido, Ioana Baldini +1
The widespread use of large language models has brought up essential questions about the potential biases these models might learn. This led to the development of several metrics a…
A Deep Dive into the Trade-Offs of Parameter-Efficient Preference Alignment Techniques
Megh Thakkar, Quentin Fournier, Matthew D Riemer +4
Large language models are first pre-trained on trillions of tokens and then instruction-tuned or aligned to specific preferences. While pre-training remains out of reach for most r…
Mastering Memory Tasks with World Models
Mohammad Reza Samsami, Artem Zholus, Janarthanan Rajendran +1
Current model-based reinforcement learning (MBRL) agents struggle with long-term dependencies. This limits their ability to effectively solve tasks involving extended time gaps bet…