1 citations · 1 across the 8 of their papers we have counts for
5 papers · 1 filter
Learning to Detect Shifts in Unstructured Data Streams via Monitoring-by-Betting
Siyu Zhou, Jialin Ouyang, Fan Xia +8
Continuous monitoring of generative AI systems requires detecting distribution shifts in the unstructured data they ingest and generate. Existing monitoring methods typically contr…
Monitoring for Shifts at Unknown Times: Parameter-Free Conditional Conformal Martingales
Tuo Liu, Drew Prinster, Francesco Orabona
Conditional conformal test martingales provide sequential tests for detecting distribution shifts, using a fixed reference dataset to avoid contamination from test-time samples. Pr…
Toward Calibrated Mixture-of-Experts Under Distribution Shift
Gina Wong, Drew Prinster, Suchi Saria +2
Calibration aligns a model's predictive uncertainty with the frequencies of its empirical outcomes and is important for understanding and trusting reported probabilities. Recent wo…
Conformal Policy Control
Drew Prinster, Clara Fannjiang, Ji Won Park +4
An agent must try new behaviors to explore and improve. In high-stakes environments, an agent that violates safety constraints may cause harm and must be taken offline, curtailing…
Testing For Distribution Shifts with Conditional Conformal Test Martingales
Shalev Shaer, Yarin Bar, Drew Prinster +1
We propose a sequential test for detecting arbitrary distribution shifts that allows conformal test martingales (CTMs) to work under a fixed, reference-conditional setting. Existin…