5 papers · 1 filter
Semiparametric Efficiency in Sequential Experiments: Characterization and Design via Average Propensity
Jiachun Li, David Simchi-Levi
Modern experiments, including evaluations of AI-enabled services and platform interventions, often depart from independent and identically distributed (i.i.d.) sampling because ass…
Low Rank for Rank: Uncertainty-Aware Task-Specific LLM Ranking under Sparse Pairwise Comparisons
Jiachun Li, David Simchi-Levi, Will Wei Sun
Pairwise human-preference platforms such as Chatbot Arena have become central to large language model (LLM) evaluation, yet reliable task-specific ranking remains challenging. Glob…
LLM Evaluation as Tensor Completion: Low Rank Structure and Semiparametric Efficiency
Jiachun Li, David Simchi-Levi, Will Wei Sun
Large language model (LLM) evaluation platforms increasingly rely on pairwise human judgments. These data are noisy, sparse, and non-uniform, yet leaderboards are reported with lim…
Beyond ATE: Multi-Criteria Design for A/B Testing
Jiachun Li, Kaining Shi, David Simchi-Levi
In the era of large-scale AI deployment and high-stakes clinical trials, adaptive experimentation faces a ``trilemma'' of conflicting objectives: minimizing cumulative regret (welf…
Privacy Preserving Adaptive Experiment Design
Jiachun Li, Kaining Shi, David Simchi-Levi
Adaptive experiment is widely adopted to estimate conditional average treatment effect (CATE) in clinical trials and many other scenarios. While the primary goal in experiment is t…