9 citations · 29 across the 18 of their papers we have counts for
19 papers
Confidence Horizons
Chase Mathis, Ian Waudby-Smith
Anytime-valid inference enables analysts to continuously monitor their data and stop experiments early. However, the majority of these methods incur a certain conservativeness by r…
Post-Training at the Edge of Detectability: A Game-Theoretic Approach to Fine-Tuning
Keegan Harris, Brian W. Lee, Ian Waudby-Smith +3
Reinforcement learning (RL) fine-tuning is widely used in language model training to improve performance on a target task while limiting drift from a reference policy. A standard w…
Gaffke's confidence interval for the mean of bounded data is inadmissible but asymptotically efficient
Jiahao Ming, Aaditya Ramdas, Yi Shen +2
Given observations , Gaffke (2005) defined \[ K_n(\mathbf x)=\mathbb{P}_{\mathbf D}\!\left\{\sum_{i=1}^n x_iD_i\le 1\right\}, \qquad (D_0,D_1,\ldots,D_n)…
Post-Hoc Large-Sample Statistical Inference
Ben Chugg, Etienne Gauthier, Michael I. Jordan +2
We derive inferential procedures for large sample sizes that remain valid under data-dependent significance levels (so-called "post-hoc valid inference"). Classical statistical too…
Multi-Armed Sequential Hypothesis Testing by Betting
Ricardo J. Sandoval, Ian Waudby-Smith, Michael I. Jordan
We consider a variant of sequential testing by betting where, at each time step, the statistician is presented with multiple data sources (arms) and obtains data by choosing one of…
Combining e-values using demi-supermartingales
Jiahao Ming, Aaditya Ramdas, Yi Shen +2
We present a new method for combining e-variables through demi-supermartingales, which settles an old conjecture in the literature on nonparametric mean testing. It also provides a…
On Nonasymptotic Confidence Intervals for Treatment Effects in Randomized Experiments
Ricardo J. Sandoval, Sivaraman Balakrishnan, Avi Feller +2
We study nonasymptotic (finite-sample) confidence intervals for treatment effects in randomized experiments. In the existing literature, the effective sample sizes of nonasymptotic…
Concentration inequalities for strong laws and laws of the iterated logarithm
Johannes Ruf, Ian Waudby-Smith
We derive concentration inequalities for sums of independent and identically distributed random variables that yield non-asymptotic generalizations of several strong laws of large…
Instance-Log-Optimality of Portfolio-Based E-Processes and their Sequential Hypothesis Tests
Ian Waudby-Smith, Ricardo Sandoval, Ricardo J. Sandoval +1
We consider the problem of sequential hypothesis testing using -processes. For a rich class of composite testing problems---which include bounded mean testing, equal mean testin…
Nonasymptotic and distribution-uniform Komlós-Major-Tusnády approximation
Ian Waudby-Smith, Martin Larsson, Aaditya Ramdas
We present nonasymptotic concentration inequalities for sums of independent and identically distributed random variables that yield asymptotic strong Gaussian approximations of Kom…
Distribution-uniform strong laws of large numbers
Ian Waudby-Smith, Martin Larsson, Aaditya Ramdas
We revisit the question of whether the strong law of large numbers (SLLN) holds uniformly in a rich family of distributions, culminating in a distribution-uniform generalization of…
Distribution-uniform anytime-valid sequential inference and the Robbins-Siegmund distributions
Ian Waudby-Smith, Edward H. Kennedy, Aaditya Ramdas
This paper develops a theory of distribution- and time-uniform asymptotics, culminating in the first large-sample anytime-valid inference procedures that are shown to be uniformly…
Anytime-Valid Confidence Sequences in an Enterprise A/B Testing Platform
Akash V. Maharaj, Ritwik Sinha, David Arbour +7
A/B tests are the gold standard for evaluating digital experiences on the web. However, traditional "fixed-horizon" statistical methods are often incompatible with the needs of mod…
Anytime-valid off-policy inference for contextual bandits
Ian Waudby-Smith, Lili Wu, Aaditya Ramdas +2
Contextual bandit algorithms are ubiquitous tools for active sequential experimentation in healthcare and the tech industry. They involve online learning algorithms that adaptively…
Nonparametric extensions of randomized response for private confidence sets
Ian Waudby-Smith, Zhiwei Steven Wu, Aaditya Ramdas
This work derives methods for performing nonparametric, nonasymptotic statistical inference for population means under the constraint of local differential privacy (LDP). Given bou…
RiLACS: Risk-Limiting Audits via Confidence Sequences
Ian Waudby-Smith, Philip B. Stark, Aaditya Ramdas
Accurately determining the outcome of an election is a complex task with many potential sources of error, ranging from software glitches in voting machines to procedural lapses to…
Time-uniform central limit theory and asymptotic confidence sequences
Ian Waudby-Smith, David Arbour, Ritwik Sinha +2
Confidence intervals based on the central limit theorem (CLT) are a cornerstone of classical statistics. Despite being only asymptotically valid, they are ubiquitous because they p…
Estimating means of bounded random variables by betting
Ian Waudby-Smith, Aaditya Ramdas
This paper derives confidence intervals (CI) and time-uniform confidence sequences (CS) for the classical problem of estimating an unknown mean from bounded observations. We presen…
Confidence sequences for sampling without replacement
Ian Waudby-Smith, Aaditya Ramdas
Many practical tasks involve sampling sequentially without replacement (WoR) from a finite population of size , in an attempt to estimate some parameter . Accurately qu…