16 papers
CalArena: A Large-Scale Post-Hoc Calibration Benchmark
Eugène Berta, David Holzmüller, Francis Bach +1
Reliable probability estimates are critical in many machine learning applications, yet modern classifiers are often poorly calibrated. Post-hoc calibration provides a simple and wi…
Conditional Coverage Diagnostics for Conformal Prediction
Sacha Braun, David Holzmüller, Michael I. Jordan +1
Evaluating conditional coverage remains one of the most persistent challenges in assessing the reliability of predictive systems. Although conformal methods can give guarantees on…
Anytime Detection of Strategic Deviations in Multi-Agent Systems
Etienne Gauthier, Francis Bach, Michael I. Jordan
In many multi-agent systems, agents interact repeatedly and are expected to settle into stable, rational behavior over time. Yet in practice, behavior often drifts, and detecting s…
Super-Level-Set Regression: Conditional Quantiles via Volume Minimization
Sacha Braun, Michael I. Jordan, Francis Bach
Constructing minimum-volume prediction regions that satisfy conditional coverage is a fundamental challenge in multivariate regression. Standard approaches rely on explicitly estim…
Multivariate Standardized Residuals for Conformal Prediction
Sacha Braun, Eugène Berta, Michael I. Jordan +1
While split conformal prediction guarantees marginal coverage, approaching the stronger property of conditional coverage is essential for reliable uncertainty quantification. Naive…
Explaining and Preventing Alignment Collapse in Iterative RLHF
Etienne Gauthier, Francis Bach, Michael I. Jordan
Reinforcement learning from human feedback (RLHF) typically assumes a static or non-strategic reward model (RM). In iterative deployment, however, the policy generates the data on…