4 papers · 1 filter
Bayesian Inference Procedures for A/B Testing: An Overview
Mårten Schultzberg, Mattias Frånberg
Bayesian inference for A/B testing is a family of prior and stopping-rule configurations with fundamentally different statistical properties, but it is often discussed as a single…
Statistical Foundations of LLM-based A/B Testing: A Surrogacy Framework for Human Causal Inference
Joel Persson, Mårten Schultzberg, Sebastian Ankargren
Organizations and researchers show increasing interest in using large language models (LLMs) in place of human participants in A/B tests, in the hope of experimenting faster and at…
Resampling-free bootstrap inference for quantiles
Mårten Schultzberg, Sebastian Ankargren
Bootstrap inference is a powerful tool for obtaining robust inference for quantiles and difference-in-quantiles estimators. The computationally intensive nature of bootstrap infere…
Statistical Properties of Exclusive and Non-exclusive Online Randomized Experiments using Bucket Reuse
Mårten Schultzberg, Oskar Kjellin, Johan Rydberg
Randomized experiments is a key part of product development in the tech industry. It is often necessary to run programs of exclusive experiments, i.e., experiments that cannot be r…