51 citations · 112 across the 28 of their papers we have counts for
32 papers
Indirect Influences of Artificial Intelligence Advice on Complex Sepsis Treatment Decisions
Venkatesh Sivaraman, Joel Levin, Eric P. Mason +6
Sepsis is a major public health concern, and many sepsis patients die due to ineffective hemodynamic management. Artificial intelligence could support hemodynamic decisions, partic…
Learning to Detect Shifts in Unstructured Data Streams via Monitoring-by-Betting
Siyu Zhou, Jialin Ouyang, Fan Xia +8
Continuous monitoring of generative AI systems requires detecting distribution shifts in the unstructured data they ingest and generate. Existing monitoring methods typically contr…
"I Know Where to Look," But Does the LLM? Charting the Gaps Between Clinical Expert Needs and Unstructured Data Abstraction Tools
Venkatesh Sivaraman, Rigney Turnham, George Bonano +16
Clinical data abstraction, the process of distilling structured information from patient records, plays a key role in advancing knowledge about diseases such as cancer. Information…
Expert-Guided g-computation with Large Language Models for Estimating Causal Effects on Timings: Applications to Hospital Quality Improvement
Patrick Vossler, Jialin Ouyang, F. Richard Guo +5
Hospital quality improvement (QI) programs routinely face multiple candidate interventions to optimize hospital flow, but existing methods struggle to estimate and rank the causal…
Expert Evaluation of Clinical AI Tools on Real Point-of-Care Clinical Queries
Jean Feng, Vishal Patel, Patrick Heagerty +5
Physicians now pose millions of clinical questions to AI tools each week, yet these tools are evaluated largely on hypothetical or exam-style questions, not those actually asked in…
Adaptive auditing of AI systems with anytime-valid guarantees
Siyu Zhou, Patrick Vossler, Venkatesh Sivaraman +2
A major bottleneck in characterizing the failure modes of generative AI systems is the cost and time of annotation and evaluation. Consequently, adaptive testing paradigms have gai…
From Fuzzy to Formal: Scaling Hospital Quality Improvement with AI
Patrick Vossler, Jean Feng, Venkat Sivaraman +9
Hospital Quality Improvement (QI) plays a critical role in optimizing healthcare delivery by translating high-level hospital goals into actionable solutions. A critical step of QI…
More Than "Means to an End": Supporting Reasoning with Transparently Designed AI Data Science Processes
Venkatesh Sivaraman, Patrick Vossler, Adam Perer +2
Generative artificial intelligence (AI) tools can now help people perform complex data science tasks regardless of their expertise. While these tools have great potential to help m…
Human-AI Co-design for Clinical Prediction Models
Jean Feng, Avni Kothari, Patrick Vossler +6
Developing safe, effective, and practically useful clinical prediction models (CPMs) traditionally requires iterative collaboration between clinical experts, data scientists, and i…
Realistic CDSS Drug Dosing with End-to-end Recurrent Q-learning for Dual Vasopressor Control
Will Y. Zou, Jean Feng, Alexandre Kalimouttou +3
Reinforcement learning (RL) applications in Clinical Decision Support Systems (CDSS) frequently encounter skepticism because models may recommend inoperable dosing decisions. We pr…
Reflections from Research Roundtables at the Conference on Health, Inference, and Learning (CHIL) 2025
Emily Alsentzer, Marie-Laure Charpignon, Bill Chen +90
The 6th Annual Conference on Health, Inference, and Learning (CHIL 2025), hosted by the Association for Health Learning and Inference (AHLI), was held in person on June 25-27, 2025…
When the Domain Expert Has No Time and the LLM Developer Has No Clinical Expertise: Real-World Lessons from LLM Co-Design in a Safety-Net Hospital
Avni Kothari, Patrick Vossler, Jean Digitale +8
Large language models (LLMs) have the potential to address social and behavioral determinants of health by transforming labor intensive workflows in resource-constrained settings.…
"Who experiences large model decay and why?" A Hierarchical Framework for Diagnosing Heterogeneous Performance Drift
Harvineet Singh, Fan Xia, Alexej Gossmann +3
Machine learning (ML) models frequently experience performance degradation when deployed in new contexts. Such degradation is rarely uniform: some subgroups may suffer large perfor…
LLMs Judging LLMs: A Simplex Perspective
Patrick Vossler, Fan Xia, Yifan Mai +2
Given the challenge of automatically evaluating free-form outputs from large language models (LLMs), an increasingly common solution is to use LLMs themselves as the judging mechan…
Bayesian Concept Bottleneck Models with LLM Priors
Jean Feng, Avni Kothari, Luke Zier +2
Concept Bottleneck Models (CBMs) have been proposed as a compromise between white-box and black-box models, aiming to achieve interpretability without sacrificing accuracy. The sta…
A hierarchical decomposition for explaining ML performance discrepancies
Jean Feng, Harvineet Singh, Fan Xia +2
Machine learning (ML) algorithms can often differ in performance across domains. Understanding their performance differs is crucial for determining what types of int…
A Brief Tutorial on Sample Size Calculations for Fairness Audits
Harvineet Singh, Fan Xia, Mi-Ok Kim +3
In fairness audits, a standard objective is to detect whether a given algorithm performs substantially differently between subgroups. Properly powering the statistical analysis of…
Designing monitoring strategies for deployed machine learning algorithms: navigating performativity through a causal lens
Jean Feng, Adarsh Subbaswamy, Alexej Gossmann +7
After a machine learning (ML)-based system is deployed, monitoring its performance is important to ensure the safety and effectiveness of the algorithm over time. When an ML algori…
Is this model reliable for everyone? Testing for strong calibration
Jean Feng, Alexej Gossmann, Romain Pirracchio +3
In a well-calibrated risk prediction model, the average predicted probability is close to the true event rate for any given subgroup. Such models are reliable across heterogeneous…
Monitoring machine learning (ML)-based risk prediction algorithms in the presence of confounding medical interventions
Jean Feng, Alexej Gossmann, Gene Pennello +3
Performance monitoring of machine learning (ML)-based risk prediction models in healthcare is complicated by the issue of confounding medical interventions (CMI): when an algorithm…
Sequential algorithmic modification with test data reuse
Jean Feng, Gene Pennello, Nicholas Petrick +3
After initial release of a machine learning algorithm, the model can be fine-tuned by retraining on subsequently gathered data, adding newly discovered features, or more. Each modi…
Bayesian logistic regression for online recalibration and revision of risk prediction models with performance guarantees
Jean Feng, Alexej Gossmann, Berkman Sahiner +1
After deploying a clinical prediction model, subsequently collected data can be used to fine-tune its predictions and adapt to temporal shifts. Because model updating carries risks…
Learning how to approve updates to machine learning algorithms in non-stationary settings
Jean Feng
Machine learning algorithms in healthcare have the potential to continually learn from real-world data generated during healthcare delivery and adapt to dataset shifts. As such, th…
Efficient nonparametric statistical inference on population feature importance using Shapley values
Brian D. Williamson, Jean Feng
The true population-level importance of a variable in a prediction task provides useful knowledge about the underlying data-generating mechanism and can help in deciding which meas…