14 citations · 16 across the 7 of their papers we have counts for
9 papers
On the Benefit of Blocking for Online Experiments with Skewed Data
Abbas Zaidi, Matthew Reimherr, Rina Friedberg +2
This paper advocates for using blocked (stratified) assignment in online A/B tests to handle highly skewed population data. While blocking yields only modest precision gains (5-10%…
Incentive Alignment in Online Experimentation
Ermis Soumalias, {Ermis Soumalias, Richard Mudd +1
Evaluating the causal effect of new features is a central goal for online platforms. While recent literature addresses limited testing traffic via centralized portfolio optimizatio…
Breaking the Winner's Curse with Bayesian Hybrid Shrinkage
Richard Mudd, Abbas Zaidi, Rina Friedberg +3
The widespread adoption of randomized controlled trials (A/B Tests) for decision-making has introduced a pervasive "Winner's Curse": experiments selected for launch often exhibit u…
Bayesian Predictive Probabilities for Online Experimentation
Abbas Zaidi, Rina Friedberg, Samir Khan +4
The widespread adoption of online randomized controlled experiments (A/B Tests) for decision-making has created ongoing capacity constraints which necessitate interim analyses. As…
Breaking the Winner's Curse with Bayesian Hybrid Shrinkage
Richard Mudd, Rina Friedberg, Ilya Gorbachev +2
A 'Winner's Curse' arises in large-scale online experimentation platforms when the same experiments are used to both select treatments and evaluate their effects. In these settings…
Treatment Effect Learning Under Sequential Randomization
Rina Friedberg, Richard Mudd, Patrick Johnstone +4
Sequential treatment assignments in online experiments lead to complex dependency structures, often rendering identification, estimation and inference over treatments a challenge.…
Active learning with biased non-response to label requests
Thomas Robinson, Niek Tax, Richard Mudd +1
Active learning can improve the efficiency of training prediction models by identifying the most informative new labels to acquire. However, non-response to label requests can impa…
TCE: A Test-Based Approach to Measuring Calibration Error
Takuo Matsubara, Niek Tax, Richard Mudd +1
This paper proposes a new metric to measure the calibration error of probabilistic binary classifiers, called test-based calibration error (TCE). TCE incorporates a novel loss func…
Explaining Predictive Uncertainty with Information Theoretic Shapley Values
David S. Watson, Joshua O'Hara, Niek Tax +2
Researchers in explainable artificial intelligence have developed numerous methods for helping users understand the predictions of complex supervised learning models. By contrast,…