18 citations · 87 across the 23 of their papers we have counts for
23 papers
Alternative statistical inference for the first normalized incomplete moment
Jiannan Lu, Peng Ding, Anqi Zhao
This paper re-examines the first normalized incomplete moment, a well-established measure of inequality with wide applications in economic and social sciences. Despite the populari…
Privacy-preserving Quantile Treatment Effect Estimation for Randomized Controlled Trials
Leon Yao, Paul Yiming Li, Jiannan Lu
In accordance with the principle of "data minimization", many internet companies are opting to record less data. However, this is often at odds with A/B testing efficacy. For exper…
All about sample-size calculations for A/B testing: Novel extensions and practical guide
Jing Zhou, Jiannan Lu, Anas Shallah
While there exists a large amount of literature on the general challenges of and best practices for trustworthy online A/B testing, there are limited studies on sample size estimat…
Feedback Effect in User Interaction with Intelligent Assistants: Delayed Engagement, Adaption and Drop-out
Zidi Xiu, Kai-Chen Cheng, David Q. Sun +7
With the growing popularity of intelligent assistants (IAs), evaluating IA quality becomes an increasingly active field of research. This paper identifies and quantifies the feedba…
Addressing Hidden Imperfections in Online Experimentation
Jeffrey Wong, Jasmine Nettiksimmons, Jiannan Lu +1
Technology companies are increasingly using randomized controlled trials (RCTs) as part of their development process. Despite having fine control over engineering systems and data…
The equivalence of the Delta method and the cluster-robust variance estimator for the analysis of clustered randomized experiments
Alex Deng, Jiannan Lu, Wen Qin
It often happens that the same problem presents itself to different communities and the solutions proposed or adopted by those communities are different. We take the case of the va…
Novelty and Primacy: A Long-Term Estimator for Online Experiments
Soheil Sadeghi, Somit Gupta, Stefan Gramatovici +3
Online experiments are the gold standard for evaluating impact on user experience and accelerating innovation in software. However, since experiments are typically limited in durat…
On Post-Selection Inference in A/B Tests
Alex Deng, Yicheng Li, Jiannan Lu +1
When interpreting A/B tests, we typically focus only on the statistically significant results and take them by face value. This practice, termed post-selection inference in the sta…
Sharp bounds on the relative treatment effect for ordinal outcomes
Jiannan Lu, Yunshu Zhang, Peng Ding
For ordinal outcomes, the average treatment effect is often ill-defined and hard to interpret. Echoing Agresti and Kateri (2017), we argue that the relative treatment effect can be…
On Heavy-user Bias in A/B Testing
Yu Wang, Somit Gupta, Jiannan Lu +2
On-line experimentation (also known as A/B testing) has become an integral part of software development. To timely incorporate user feedback and continuously improve products, many…
Applying the Delta method in metric analytics: A practical guide with novel ideas
Alex Deng, Ulf Knoblich, Jiannan Lu
During the last decade, the information technology industry has adopted a data-driven culture, relying on online metrics to measure and monitor business performance. Under the sett…
Improved Neymanian analysis for factorial designs with binary outcomes
Jiannan Lu
factorial designs are widely adopted by statisticians and the broader scientific community. In this short note, under the potential outcomes framework (Neyman, 1923; Rubin, 1…
On finite-population Bayesian inferences for factorial designs with binary outcomes
Jiannan Lu
Inspired by the pioneering work of Rubin (1978), we employ the potential outcomes framework to develop a finite-population Bayesian causal inference framework for randomized contro…
Sharpening randomization-based causal inference for factorial designs with binary outcomes
Jiannan Lu
In medical research, a scenario often entertained is randomized controlled factorial design with a binary outcome. By utilizing the concept of potential outcomes, Dasgupta et…
On randomization-based causal inference for matched-pair factorial designs
Jiannan Lu, Alex Deng
Under the potential outcomes framework, we introduce matched-pair factorial designs, and propose the matched-pair estimator of the factorial effects. We also calculate the randomiz…
Concise Summarization of Heterogeneous Treatment Effect Using Total Variation Regularized Regression
Alex Deng, Pengchuan Zhang, Shouyuan Chen +2
Randomized controlled experiment has long been accepted as the golden standard for establishing causal link and estimating causal effect in various scientific fields. Average treat…
Covariate adjustment in randomization-based causal inference for 2K factorial designs
Jiannan Lu
We develop finite-population asymptotic theory for covariate adjustment in randomization-based causal inference for 2K factorial designs. In particular, we confirm that both the un…
Continuous Monitoring of A/B Tests without Pain: Optional Stopping in Bayesian Testing
Alex Deng, Jiannan Lu, Shouyuan Chen
A/B testing is one of the most successful applications of statistical theory in modern Internet age. One problem of Null Hypothesis Statistical Testing (NHST), the backbone of A/B…
On Randomization-based and Regression-based Inferences for 2^K Factorial Designs
Jiannan Lu
We extend the randomization-based causal inference framework in Dasgupta et al. (2015) for general 2^K factorial designs, and demonstrate the equivalence between regression-based a…
Principal stratification analysis using principal scores
Peng Ding, Jiannan Lu
Practitioners are interested in not only the average causal effect of the treatment on the outcome but also the underlying causal mechanism in the presence of an intermediate varia…
Demystifying the Bias from Selective Inference: a Revisit to Dawid's Treatment Selection Problem
Jiannan Lu, Alex Deng
We extend the heuristic discussion in Senn (2008) on the bias from selective inference for the treatment selection problem (Dawid 1994), by deriving the closed-form expression for…
Treatment Effects on Ordinal Outcomes: Causal Estimands and Sharp Bounds
Jiannan Lu, Peng Ding, Tirthankar Dasgupta
Assessing the causal effects of interventions on ordinal outcomes is an important objective of many educational and behavioral studies. Under the potential outcomes framework, we c…
Construction of alternative hypotheses for randomization tests with ordinal outcomes
Jiannan Lu, Peng Ding, Tirthankar Dasgupta
For ordinal outcomes, we construct sequences of alternative hypotheses in increasing departures from the sharp null hypothesis of zero treatment effect on each experimental unit, t…