5 papers
BACON: Budgeted Human Calibration for Modeling and Evaluation with Multiple AI Judges
Lei Shi, Anlan Zhang, Rita Lyu +6
AI judges offer a scalable, low-cost alternative to human evaluation, but their outputs can be biased relative to human preferences and highly item-dependent, varying across judges…
Experimentation Accelerator: Interpretable Insights and Creative Recommendations for A/B Testing with Content-Aware ranking
Zhengmian Hu, Lei Shi, Ritwik Sinha +2
Modern online experimentation faces two bottlenecks: scarce traffic forces tough choices on which variants to test, and post-hoc insight extraction is manual, inconsistent, and oft…
Leveraging semantic similarity for experimentation with AI-generated treatments
Lei Shi, David Arbour, Raghavendra Addanki +2
Large Language Models (LLMs) enable a new form of digital experimentation where treatments combine human and model-generated content in increasingly sophisticated ways. The main me…
Continuous Treatment Effects with Surrogate Outcomes
Zhenghao Zeng, David Arbour, Avi Feller +4
In many real-world causal inference applications, the primary outcomes (labels) are often partially missing, especially if they are expensive or difficult to collect. If the missin…
Augment before You Try: Knowledge-Enhanced Table Question Answering via Table Expansion
Yujian Liu, Jiabao Ji, Tong Yu +6
Table question answering is a popular task that assesses a model's ability to understand and interact with structured data. However, the given table often does not contain sufficie…