Designing Persuasive Experiments

Abstract

Incentives in experimental design are often misaligned: experimenters design and finance experiments to seek regulatory approval, while regulators seek to maximize social welfare. This incentive misalignment is especially relevant for Bayesian and adaptive trial designs, which have gained substantial popularity in recent years and have been actively encouraged by the US Food and Drug Administration. We propose a framework to resolve this conflict, wherein regulators set a minimum welfare threshold, and experimenters optimize designs subject to this constraint. It requires no knowledge of experimenters' private preferences or costs and mitigates strategic Bayesian persuasion. By varying the welfare threshold, we can map the Pareto frontier between the experimenter's and regulator's welfare. Under normal priors, Neyman allocation is always the optimal sampling strategy, regardless of specific objectives. We also characterize the optimal stopping rule. A numerical study calibrated to clinical trial data shows sample size reductions of over 48% relative to classical designs attaining the same social welfare.