8 papers
Consistent but Miscalibrated: Evaluating LLM Limitations for Risk Communication in Natural Language
Diego Cerda-Mardini, Sarath Chandar, Sreenath Madathil
LLMs are increasingly deployed as post-hoc explainers of AI-generated outputs, yet it remains unclear whether they can reliably communicate probabilistic information in natural lan…
Failed Reasoning Traces Tell You What Is Fixable (But Not by Reading Them)
Nizar Islah, Istabrak Abbes, Irina Rish +2
When post-trained language models fail on reasoning problems, the common test-time-scaling response is to spend more compute on additional attempts, and the failed traces play no f…
Probabilistic Calibration Is a Trainable Capability in Language Models
Davide Baldelli, Sruthi Kuriakose, Maryam Hashemzadeh +2
Language models are increasingly used in settings where outputs must satisfy user-specified randomness constraints, yet their generation probabilities are often poorly calibrated t…
Exploring Quantization for Efficient Pre-Training of Transformer Language Models
Kamran Chitsaz, Quentin Fournier, Gonçalo Mordido +1
The increasing scale of Transformer models has led to an increase in their pre-training computational requirements. While quantization has proven to be effective after pre-training…
Why Don't Prompt-Based Fairness Metrics Correlate?
Abdelrahman Zayed, Goncalo Mordido, Ioana Baldini +1
The widespread use of large language models has brought up essential questions about the potential biases these models might learn. This led to the development of several metrics a…
A Deep Dive into the Trade-Offs of Parameter-Efficient Preference Alignment Techniques
Megh Thakkar, Quentin Fournier, Matthew D Riemer +4
Large language models are first pre-trained on trillions of tokens and then instruction-tuned or aligned to specific preferences. While pre-training remains out of reach for most r…