18 citations · 87 across the 23 of their papers we have counts for
6 papers · 1 filter
On Post-Selection Inference in A/B Tests
Alex Deng, Yicheng Li, Jiannan Lu +1
When interpreting A/B tests, we typically focus only on the statistically significant results and take them by face value. This practice, termed post-selection inference in the sta…
On Heavy-user Bias in A/B Testing
Yu Wang, Somit Gupta, Jiannan Lu +2
On-line experimentation (also known as A/B testing) has become an integral part of software development. To timely incorporate user feedback and continuously improve products, many…
Applying the Delta method in metric analytics: A practical guide with novel ideas
Alex Deng, Ulf Knoblich, Jiannan Lu
During the last decade, the information technology industry has adopted a data-driven culture, relying on online metrics to measure and monitor business performance. Under the sett…
Concise Summarization of Heterogeneous Treatment Effect Using Total Variation Regularized Regression
Alex Deng, Pengchuan Zhang, Shouyuan Chen +2
Randomized controlled experiment has long been accepted as the golden standard for establishing causal link and estimating causal effect in various scientific fields. Average treat…
Continuous Monitoring of A/B Tests without Pain: Optional Stopping in Bayesian Testing
Alex Deng, Jiannan Lu, Shouyuan Chen
A/B testing is one of the most successful applications of statistical theory in modern Internet age. One problem of Null Hypothesis Statistical Testing (NHST), the backbone of A/B…
Construction of alternative hypotheses for randomization tests with ordinal outcomes
Jiannan Lu, Peng Ding, Tirthankar Dasgupta
For ordinal outcomes, we construct sequences of alternative hypotheses in increasing departures from the sharp null hypothesis of zero treatment effect on each experimental unit, t…