1 citations · 2 across the 6 of their papers we have counts for
7 papers
Learning to Detect Shifts in Unstructured Data Streams via Monitoring-by-Betting
Siyu Zhou, Jialin Ouyang, Fan Xia +8
Continuous monitoring of generative AI systems requires detecting distribution shifts in the unstructured data they ingest and generate. Existing monitoring methods typically contr…
Expert-Guided g-computation with Large Language Models for Estimating Causal Effects on Timings: Applications to Hospital Quality Improvement
Patrick Vossler, Jialin Ouyang, F. Richard Guo +5
Hospital quality improvement (QI) programs routinely face multiple candidate interventions to optimize hospital flow, but existing methods struggle to estimate and rank the causal…
"Who experiences large model decay and why?" A Hierarchical Framework for Diagnosing Heterogeneous Performance Drift
Harvineet Singh, Fan Xia, Alexej Gossmann +3
Machine learning (ML) models frequently experience performance degradation when deployed in new contexts. Such degradation is rarely uniform: some subgroups may suffer large perfor…
LLMs Judging LLMs: A Simplex Perspective
Patrick Vossler, Fan Xia, Yifan Mai +2
Given the challenge of automatically evaluating free-form outputs from large language models (LLMs), an increasingly common solution is to use LLMs themselves as the judging mechan…
A hierarchical decomposition for explaining ML performance discrepancies
Jean Feng, Harvineet Singh, Fan Xia +2
Machine learning (ML) algorithms can often differ in performance across domains. Understanding their performance differs is crucial for determining what types of int…
A Brief Tutorial on Sample Size Calculations for Fairness Audits
Harvineet Singh, Fan Xia, Mi-Ok Kim +3
In fairness audits, a standard objective is to detect whether a given algorithm performs substantially differently between subgroups. Properly powering the statistical analysis of…
Designing monitoring strategies for deployed machine learning algorithms: navigating performativity through a causal lens
Jean Feng, Adarsh Subbaswamy, Alexej Gossmann +7
After a machine learning (ML)-based system is deployed, monitoring its performance is important to ensure the safety and effectiveness of the algorithm over time. When an ML algori…