6 papers
Trained on Tokens, Calibrated on Concepts: The Emergence of Semantic Calibration in LLMs
Preetum Nakkiran, Arwen Bradley, Adam Goliński +3
Large Language Models (LLMs) often lack meaningful confidence estimates for their outputs. While base LLMs are known to exhibit next-token calibration, it remains unclear whether t…
The Geometries of Truth Are Orthogonal Across Tasks
Waiss Azizian, Michael Kirchhof, Eugene Ndiaye +4
Large Language Models (LLMs) have demonstrated impressive generalization capabilities across various tasks, but their claim to practical relevance is still mired by concerns on the…
Multivariate Conformal Prediction using Optimal Transport
Michal Klein, Louis Bethune, Eugene Ndiaye +1
Conformal prediction (CP) quantifies the uncertainty of machine learning models by constructing sets of plausible outputs. These sets are constructed by leveraging a so-called conf…
Shielded Diffusion: Generating Novel and Diverse Images using Sparse Repellency
Michael Kirchhof, James Thornton, Louis Béthune +3
The adoption of text-to-image diffusion models raises concerns over reliability, drawing scrutiny under the lens of various metrics like calibration, fairness, or compute efficienc…
Careful with that Scalpel: Improving Gradient Surgery with an EMA
Yu-Guan Hsieh, James Thornton, Eugene Ndiaye +3
Beyond minimizing a single training loss, many deep learning estimation pipelines rely on an auxiliary objective to quantify and encourage desirable properties of the model (e.g. p…
Exact and Approximate Conformal Inference for Multi-Output Regression
Chancellor Johnstone, Eugene Ndiaye
It is common in machine learning to estimate a response given covariate information . However, these predictions alone do not quantify any uncertainty associated with said p…