On the Benefit of Blocking for Online Experiments with Skewed Data

Abbas Zaidi, Matthew Reimherr, Rina Friedberg +2

This paper advocates for using blocked (stratified) assignment in online A/B tests to handle highly skewed population data. While blocking yields only modest precision gains (5-10%…