5 papers
Clustering-based Imputation for Dropout Buyers in Large-scale Online Experimentation
Sumin Shen, Huiying Mao, Zezhong Zhang +3
In online experimentation, appropriate metrics (e.g., purchase) provide strong evidence to support hypotheses and enhance the decision-making process. However, incomplete metrics a…
Ensure A/B Test Quality at Scale with Automated Randomization Validation and Sample Ratio Mismatch Detection
Keyu Nie, Zezhong Zhang, Bingquan Xu +1
eBay's experimentation platform runs hundreds of A/B tests on any given day. The platform integrates with the tracking infrastructure and customer experience servers, provides the…
Moving Metric Detection and Alerting System at eBay
Zezhong Zhang, Keyu Nie, Ted Tao Yuan
At eBay, there are thousands of product health metrics for different domain teams to monitor. We built a two-phase alerting system to notify users with actionable alerts based on a…
Dealing With Ratio Metrics in A/B Testing at the Presence of Intra-User Correlation and Segments
Keyu Nie, Yinfei Kong, Ted Tao Yuan +1
We study ratio metrics in A/B testing at the presence of correlation among observations coming from the same user and provides practical guidance especially when two metrics contra…
Efficient Multivariate Bandit Algorithm with Path Planning
Keyu Nie, Zezhong Zhang, Ted Tao Yuan +2
In this paper, we solve the arms exponential exploding issue in multivariate Multi-Armed Bandit (Multivariate-MAB) problem when the arm dimension hierarchy is considered. We propos…