16 papers
Eigenvalue Calibration for Semantic Embeddings of Large Language Models
Sebastian G. Gruber, Nassim Walha, Francis Bach +1
Uncertainty quantification is central to the reliable deployment of large language models (LLMs), and eigenvalues of semantic embeddings have recently emerged as a key tool in stat…
DecompRL: Solving Harder Problems by Learning Modular Code Generation
Juliette Decugis, Fabian Gloeckle, Francis Bach +2
How can Large Language Models (LLMs) solve problems they currently cannot? Repeated sampling scales test-time compute but GPU cost grows linearly with attempts, while reinforcement…
CalArena: A Large-Scale Post-Hoc Calibration Benchmark
Eugène Berta, David Holzmüller, Francis Bach +1
Reliable probability estimates are critical in many machine learning applications, yet modern classifiers are often poorly calibrated. Post-hoc calibration provides a simple and wi…
Conditional Coverage Diagnostics for Conformal Prediction
Sacha Braun, David Holzmüller, Michael I. Jordan +1
Evaluating conditional coverage remains one of the most persistent challenges in assessing the reliability of predictive systems. Although conformal methods can give guarantees on…
Anytime Detection of Strategic Deviations in Multi-Agent Systems
Etienne Gauthier, Francis Bach, Michael I. Jordan
In many multi-agent systems, agents interact repeatedly and are expected to settle into stable, rational behavior over time. Yet in practice, behavior often drifts, and detecting s…
Super-Level-Set Regression: Conditional Quantiles via Volume Minimization
Sacha Braun, Michael I. Jordan, Francis Bach
Constructing minimum-volume prediction regions that satisfy conditional coverage is a fundamental challenge in multivariate regression. Standard approaches rely on explicitly estim…