8 papers
Online Conformal Prediction Beyond Feedback
Joar Skalse, Edoardo Pona, Osvaldo Simeone +1
Uncertainty quantification is essential when deploying machine learning models in safety-critical applications. Online conformal prediction (OCP) provides theoretically principled…
Statistically Valid Post-Training Hyperparameter Selection: From Tuning to Guarantees
Amirmohammad Farzaneh, Osvaldo Simeone
Post-training hyperparameter selection is a critical step in the deployment of modern artificial intelligence systems, given the need to tune degrees of freedom of pre-trained mode…
Online Conformal Prediction with Corrupted Feedback
Bowen Wang, Matteo Zecchin, Osvaldo Simeone
Modern artificial intelligence systems require calibrated uncertainty estimates that remain reliable in sequential and non-stationary environments. Online conformal prediction (OCP…
Federated Martingale Posterior Samping
Boning Zhang, Matteo Zecchin, Mingzhao Guo +2
Federated Bayesian neural networks require fixing a prior on the model parameters together with a likelihood. Eliciting meaningful priors on the weight space of modern overparamete…
Calibrate-Then-Delegate: Safety Monitoring with Risk and Budget Guarantees via Model Cascades
Edoardo Pona, Milad Kazemi, Mehran Hosseini +4
Monitoring LLM safety at scale requires balancing cost and accuracy: a cheap latent-space probe can screen every input, but hard cases should be escalated to a more expensive exper…
Beyond Fixed False Discovery Rates: Post-Hoc Conformal Selection with E-Variables
Meiyi Zhu, Osvaldo Simeone
Conformal selection (CS) uses calibration data to identify test inputs whose unobserved outcomes are likely to satisfy a pre-specified minimal quality requirement, while controllin…