9 citations · 20 across the 9 of their papers we have counts for
7 papers · 1 filter
Dependence-Aware Label Aggregation for LLM-as-a-Judge via Ising Models
Krishnakumar Balasubramanian, Aleksandr Podkopaev, Shiva Prasad Kasiviswanathan
Large-scale AI evaluation increasingly relies on aggregating binary judgments from annotators, including LLMs used as judges. Most classical methods, e.g., Dawid-Skene or (weig…
Adaptive Conformal Inference by Betting
Aleksandr Podkopaev, Darren Xu, Kuang-Chih Lee
Conformal prediction is a valuable tool for quantifying predictive uncertainty of machine learning models. However, its applicability relies on the assumption of data exchangeabili…
Sequential Predictive Two-Sample and Independence Testing
Aleksandr Podkopaev, Aaditya Ramdas
We study the problems of sequential nonparametric two-sample and independence testing. Sequential tests process data online and allow using observed data to decide whether to stop…
Sequential Kernelized Independence Testing
Aleksandr Podkopaev, Patrick Blöbaum, Shiva Prasad Kasiviswanathan +1
Independence testing is a classical statistical problem that has been extensively studied in the batch setting when one fixes the sample size before collecting data. However, pract…
Tracking the risk of a deployed model and detecting harmful distribution shifts
Aleksandr Podkopaev, Aaditya Ramdas
When deployed in the real world, machine learning models inevitably encounter changes in the data distribution, and certain -- but not all -- distribution shifts could result in si…
Distribution-free uncertainty quantification for classification under label shift
Aleksandr Podkopaev, Aaditya Ramdas
Trustworthy deployment of ML models requires a proper measure of uncertainty, especially in safety-critical applications. We focus on uncertainty quantification (UQ) for classifica…
Distribution-free binary classification: prediction sets, confidence intervals and calibration
Chirag Gupta, Aleksandr Podkopaev, Aaditya Ramdas
We study three notions of uncertainty quantification -- calibration, confidence intervals and prediction sets -- for binary classification in the distribution-free setting, that is…