3 papers · 1 filter
Dependence-Aware Label Aggregation for LLM-as-a-Judge via Ising Models
Krishnakumar Balasubramanian, Aleksandr Podkopaev, Shiva Prasad Kasiviswanathan
Large-scale AI evaluation increasingly relies on aggregating binary judgments from annotators, including LLMs used as judges. Most classical methods, e.g., Dawid-Skene or (weig…
Sequential Kernelized Independence Testing
Aleksandr Podkopaev, Patrick Blöbaum, Shiva Prasad Kasiviswanathan +1
Independence testing is a classical statistical problem that has been extensively studied in the batch setting when one fixes the sample size before collecting data. However, pract…
Adaptive Conformal Inference by Betting
Aleksandr Podkopaev, Darren Xu, Kuang-Chih Lee
Conformal prediction is a valuable tool for quantifying predictive uncertainty of machine learning models. However, its applicability relies on the assumption of data exchangeabili…