10 papers
Efficient Sequential Evaluation of Large Language Models
Chia-Yu Hsu, Shubhanshu Shekhar
We study the problem of sequentially evaluating a new large language model (LLM) on a fixed question set using historical performance data from prior LLMs. Our goal is to construct…
A Semi-Supervised Kernel Two-Sample Test
Gyumin Lee, Shubhanshu Shekhar, Ilmun Kim
We consider the problem of two-sample testing in a semi-supervised setting with abundant unlabeled covariate data. Standard two-sample tests neglect covariate information, which ha…
Dual Representation of Minimum Divergence Under Integral Constraints
Shubhanshu Shekhar, Shubhada Agrawal
Minimum divergence problems under integral constraints appear throughout statistics and probability, including sequential inference, bandit theory, and distributionally robust opti…
Classifier-Based Nonparametric Sequential Hypothesis Testing
Chia-Yu Hsu, Shubhanshu Shekhar
We consider the problem of constructing sequential power-one tests where the null and alternative classes are specified indirectly through historical or offline data. More specific…
Tighter Confidence Intervals under Without Replacement Sampling via Empirical Rate Functions
Shubhanshu Shekhar, Aaditya Ramdas
We consider the problem of constructing confidence intervals (CIs) for the population mean of values based on a random sample of size , de…
Learning to Bet for Horizon-Aware Anytime-Valid Testing
Ege Onur Taga, Samet Oymak, Shubhanshu Shekhar
We develop horizon-aware anytime-valid tests and confidence sequences for bounded means under a strict deadline . Using the betting/e-process framework, we cast horizon-aware be…
VINA: Variational Invertible Neural Architectures
Shubhanshu Shekhar, Mohammad Javad Khojasteh, Ananya Acharya +2
The distinctive architectural features of normalizing flows (NFs), notably bijectivity and tractable Jacobians, make them well-suited for generative modeling. Invertible neural net…
Outage Identification from Electricity Market Data: Quickest Change Detection Approach
Milad Hoseinpour, Shubhanshu Shekhar, Vladimir Dvorkin
Power system outages expose market participants to significant financial risk unless promptly detected and hedged. We develop an outage identification method from public market sig…
Active Nonparametric Two-Sample Testing by Betting on Heterogeneous Data Sources
Chia-Yu Hsu, Shubhanshu Shekhar
We study the problem of active nonparametric sequential two-sample testing over multiple heterogeneous data sources. In each time slot, a decision-maker adaptively selects one of $…
Optimal Anytime-Valid Tests for Composite Nulls
Shubhanshu Shekhar
We consider the problem of designing optimal level- power-one tests for composite nulls. Given a parameter and a stream of -valued observations $\{X_n:…