34 citations · 34 across the 7 of their papers we have counts for
8 papers · 1 filter
Testing for trends in online experiments
Chris Haulk, Lee Richardson, Jacopo Soriano
Time-varying treatment effects can be estimated with linear regression paired with jackknife confidence intervals. In experiments at a large online video platform, the test based o…
Choosing a Proxy Metric from Past Experiments
Nilesh Tripuraneni, Lee Richardson, Alexander D'Amour +2
In many randomized experiments, the treatment effect of the long-term metric (i.e. the primary outcome of interest) is often difficult or infeasible to measure. Such long-term metr…
Pareto optimal proxy metrics
Alessandro Zito, Dylan Greaves, Jacopo Soriano +1
North star metrics and online experimentation play a central role in how technology companies improve their products. In many practical settings, however, evaluating experiments ba…
Percent Change Estimation in Large Scale Online Experiments
Jacopo Soriano
Online experiments are a fundamental component of the development of web-facing products. Given their large user-bases, even small product improvements can have a large impact on u…
Mixture modeling on related samples by -stick breaking and kernel perturbation
Jacopo Soriano, Li Ma
There has been great interest recently in applying nonparametric kernel mixtures in a hierarchical manner to model multiple related data samples jointly. In such settings several d…
Analysis of distributional variation through multi-scale Beta-Binomial modeling
Li Ma, Jacopo Soriano
Many statistical analyses involve the comparison of multiple data sets collected under different conditions in order to identify the difference in the underlying distributions. A c…
Efficient functional ANOVA through wavelet-domain Markov groves
Li Ma, Jacopo Soriano
We introduce a wavelet-domain functional analysis of variance (fANOVA) method based on a Bayesian hierarchical model. The factor effects are modeled through a spike-and-slab mixtur…
Multi-resolution two-sample comparison through the divide-merge Markov tree
Jacopo Soriano, Li Ma
We introduce a probabilistic framework for two-sample comparison based on a nonparametric process taking the form of a Markov model that transitions between a "divide" and a "merge…