10 citations · 15 across the 11 of their papers we have counts for
11 papers
Bridging Predictions and Interventions: An Integrated Framework for Automated Decision-Systems
Inioluwa Deborah Raji, Lydia T. Liu, Angela Zhou +27
Automated decision systems (ADS) leverage predictions about individual future outcomes to inform consequential decision-making in organizational settings. Across various settings -…
Doubly-Robust LLM-as-a-Judge: Externally Valid Estimation with Imperfect Personas
Luke Guerdan, Justin Whitehouse, Kimberly Truong +2
As Generative AI (GenAI) systems see growing adoption, a key concern involves the external validity of evaluations, or the extent to which they generalize from lab-based to real-wo…
Measurement as Bricolage: Examining How Data Scientists Construct Target Variables for Predictive Modeling Tasks
Luke Guerdan, Devansh Saxena, Stevie Chancellor +2
Data scientists often formulate predictive modeling tasks involving fuzzy, hard-to-define concepts, such as the "authenticity" of student writing or the "healthcare need" of a pati…
Bridging Prediction and Intervention Problems in Social Systems
Lydia T. Liu, Inioluwa Deborah Raji, Angela Zhou +32
Many automated decision systems (ADS) are designed to solve prediction problems -- where the goal is to learn patterns from a sample of the population and apply them to individuals…
Validating LLM-as-a-Judge Systems under Rating Indeterminacy
Luke Guerdan, Solon Barocas, Kenneth Holstein +3
The LLM-as-a-judge paradigm, in which a judge LLM system replaces human raters in rating the outputs of other generative AI (GenAI) systems, plays a critical role in scaling and st…
A Framework for Evaluating LLMs Under Task Indeterminacy
Luke Guerdan, Hanna Wallach, Solon Barocas +1
Large language model (LLM) evaluations often assume there is a single correct response -- a gold label -- for each item in the evaluation corpus. However, some tasks can be ambiguo…
Predictive Performance Comparison of Decision Policies Under Confounding
Luke Guerdan, Amanda Coston, Kenneth Holstein +1
Predictive models are often introduced to decision-making tasks under the rationale that they improve performance over an existing decision-making policy. However, it is challengin…
Training Towards Critical Use: Learning to Situate AI Predictions Relative to Human Knowledge
Anna Kawakami, Luke Guerdan, Yanghuidi Cheng +6
A growing body of research has explored how to support humans in making better use of AI-based decision support, including via training and onboarding. Existing research has focuse…
Counterfactual Prediction Under Outcome Measurement Error
Luke Guerdan, Amanda Coston, Kenneth Holstein +1
Across domains such as medicine, employment, and criminal justice, predictive models often target labels that imperfectly reflect the outcomes of interest to experts and policymake…
Ground(less) Truth: A Causal Framework for Proxy Labels in Human-Algorithm Decision-Making
Luke Guerdan, Amanda Coston, Zhiwei Steven Wu +1
A growing literature on human-AI decision-making investigates strategies for combining human judgment with statistical models to improve decision-making. Research in this area ofte…
Towards a Learner-Centered Explainable AI: Lessons from the learning sciences
Anna Kawakami, Luke Guerdan, Yang Cheng +8
In this short paper, we argue for a refocusing of XAI around human learning goals. Drawing upon approaches and theories from the learning sciences, we propose a framework for the l…