2 citations · 2 across the 9 of their papers we have counts for
13 papers
Neyman Orthogonal Activation Steering
David Arbour, Erfan Jahanparast, Serina Chang +1
Activation steering is a popular approach for shifting LLM output toward a desired behavior, and increasingly for generating realistic synthetic data. The standard recipe estimates…
Always-On Experimentation
Ricardo J. Sandoval, David Arbour, Avi Feller +1
Generative AI has dramatically accelerated the rate at which new treatments---from novel pharmaceuticals to online marketing campaigns---can be conceived and deployed. As a result,…
BACON: Budgeted Human Calibration for Modeling and Evaluation with Multiple AI Judges
Lei Shi, Anlan Zhang, Rita Lyu +6
AI judges offer a scalable, low-cost alternative to human evaluation, but their outputs can be biased relative to human preferences and highly item-dependent, varying across judges…
Bridging Predictions and Interventions: An Integrated Framework for Automated Decision-Systems
Inioluwa Deborah Raji, Lydia T. Liu, Angela Zhou +27
Automated decision systems (ADS) leverage predictions about individual future outcomes to inform consequential decision-making in organizational settings. Across various settings -…
AI-Assisted Variance Reduction in Randomized Experiments
David Arbour, Eli Ben-Michael, Avi Feller +2
Generative AI and large language models can produce realistic predictions of human behavior from rich, unstructured inputs with little to no task-specific training data. Recent wor…
A Weighting Framework for Clusters as Confounders in Observational Studies
Eli Ben-Michael, Avi Feller, Luke Keele
When units in observational studies are clustered in groups, such as students in schools or patients in hospitals, researchers often address confounding by adjusting for cluster-le…
Forest Kernel Balancing Weights: Outcome-Guided Features for Causal Inference
Andy A. Shen, Eli Ben-Michael, Avi Feller +2
While balancing covariates between groups is central for observational causal inference, selecting which features to balance remains a challenging problem. Kernel balancing is a pr…
Leveraging semantic similarity for experimentation with AI-generated treatments
Lei Shi, David Arbour, Raghavendra Addanki +2
Large Language Models (LLMs) enable a new form of digital experimentation where treatments combine human and model-generated content in increasingly sophisticated ways. The main me…
Handling Missing Responses under Cluster Dependence with Applications to Language Model Evaluation
Zhenghao Zeng, David Arbour, Avi Feller +3
Human annotations play a crucial role in evaluating the performance of GenAI models. Two common challenges in practice, however, are missing annotations (the response variable of i…
Regularizing Extrapolation in Causal Inference
David Arbour, Harsh Parikh, Bijan Niknam +3
Many common estimators in machine learning and causal inference are linear smoothers, where the prediction is a weighted average of the training outcomes. Some estimators, such as…
Bridging Prediction and Intervention Problems in Social Systems
Lydia T. Liu, Inioluwa Deborah Raji, Angela Zhou +32
Many automated decision systems (ADS) are designed to solve prediction problems -- where the goal is to learn patterns from a sample of the population and apply them to individuals…
Evaluation and Incident Prevention in an Enterprise AI Assistant
Akash V. Maharaj, David Arbour, Daniel Lee +6
Enterprise AI Assistants are increasingly deployed in domains where accuracy is paramount, making each erroneous output a potentially significant incident. This paper presents a co…
Continuous Treatment Effects with Surrogate Outcomes
Zhenghao Zeng, David Arbour, Avi Feller +4
In many real-world causal inference applications, the primary outcomes (labels) are often partially missing, especially if they are expensive or difficult to collect. If the missin…