7 papers
Accelerating A/B-Tests with Counterfactual Estimation: Reducing Variance through Policy Overlap
Olivier Jeunen
Online controlled experiments are the gold standard for hypothesis testing in online platforms. Notwithstanding their ubiquity, they are notoriously expensive to run, and issues of…
Auditing Marketing Budget Allocation with Hindsight Regret
Nilavra Pathak, Olivier Jeunen, Eric Lambert
Organizations routinely make strategic budget allocations under operational constraints, but often lack a principled way to assess whether realized allocations were close to the be…
Sustained Impact of Agentic Personalisation in Marketing: A Longitudinal Case Study
Olivier Jeunen, Eleanor Hanna, Schaun Wheeler
In consumer applications, Customer Relationship Management (CRM) has traditionally relied on the manual optimisation of static, rule-based messaging strategies. While adaptive and…
Unifying On- and Off-Policy Variance Reduction Methods
Olivier Jeunen
Continuous and efficient experimentation is key to the practical success of user-facing applications on the web, both through online A/B-tests and off-policy evaluation. Despite th…
Additive Control Variates Dominate Self-Normalisation in Off-Policy Evaluation
Olivier Jeunen, Shashank Gupta
Off-policy evaluation (OPE) is essential for assessing ranking and recommendation systems without costly online interventions. Self-Normalised Inverse Propensity Scoring (SNIPS) is…
-Testing the Waters: Empirically Validating Assumptions for Reliable A/B-Testing
Olivier Jeunen
A/B-tests are a cornerstone of experimental design on the web, with wide-ranging applications and use-cases. The statistical -test comparing differences in means is the most com…