5 papers
On the Benefit of Blocking for Online Experiments with Skewed Data
Abbas Zaidi, Matthew Reimherr, Rina Friedberg +2
This paper advocates for using blocked (stratified) assignment in online A/B tests to handle highly skewed population data. While blocking yields only modest precision gains (5-10%…
Breaking the Winner's Curse with Bayesian Hybrid Shrinkage
Richard Mudd, Abbas Zaidi, Rina Friedberg +3
The widespread adoption of randomized controlled trials (A/B Tests) for decision-making has introduced a pervasive "Winner's Curse": experiments selected for launch often exhibit u…
Bayesian Predictive Probabilities for Online Experimentation
Abbas Zaidi, Rina Friedberg, Samir Khan +4
The widespread adoption of online randomized controlled experiments (A/B Tests) for decision-making has created ongoing capacity constraints which necessitate interim analyses. As…
Breaking the Winner's Curse with Bayesian Hybrid Shrinkage
Richard Mudd, Rina Friedberg, Ilya Gorbachev +2
A 'Winner's Curse' arises in large-scale online experimentation platforms when the same experiments are used to both select treatments and evaluate their effects. In these settings…
Treatment Effect Learning Under Sequential Randomization
Rina Friedberg, Richard Mudd, Patrick Johnstone +4
Sequential treatment assignments in online experiments lead to complex dependency structures, often rendering identification, estimation and inference over treatments a challenge.…