4 papers · 1 filter
Estimating the True Effect Size Distribution with SIMEX
Zhaoqi Li, Daniel Ting, Ilya Gorbachev +2
Large-scale online experimentation produces noisy effect estimates, which can overstate gains and complicate decisions about launches and testing policies. We propose a nonparametr…
Breaking the Winner's Curse with Bayesian Hybrid Shrinkage
Richard Mudd, Abbas Zaidi, Rina Friedberg +3
The widespread adoption of randomized controlled trials (A/B Tests) for decision-making has introduced a pervasive "Winner's Curse": experiments selected for launch often exhibit u…
Evaluating Variance Estimates with Relative Efficiency
Kedar Karhadkar, Jack Klys, Daniel Ting +2
Experimentation platforms in industry must often deal with customer trust issues. Platforms must prove the validity of their claims as well as catch issues that arise. As a central…
Breaking the Winner's Curse with Bayesian Hybrid Shrinkage
Richard Mudd, Rina Friedberg, Ilya Gorbachev +2
A 'Winner's Curse' arises in large-scale online experimentation platforms when the same experiments are used to both select treatments and evaluate their effects. In these settings…