2 citations · 5 across the 13 of their papers we have counts for
13 papers
Characterizing Rhetorical Misalignment in Decision-Making with Language Models
Zirui Cheng, Joey Chan, Simo Du +3
Human decision-making is often shaped by a range of well-documented cognitive biases. As large language models (LLMs) become increasingly integrated into high-stakes human-AI decis…
Benchmarking Agentic Review Systems
Dang Nguyen, Wanqing Hao, Yanai Elazar +1
A new class of agentic review systems are emerging as a remedy to the pressure placed on peer review systems by AI-assisted research, but it is unclear how they should be evaluated…
Collaborative Disagreement Resolution for Scalable Oversight
Yuyang Jiang, Chacha Chen, Teng Wu +4
Debate, where AI agents argue opposing positions, has emerged as a key approach to scalable oversight. However, debate faces a fundamental tension: models are incentivized to be pe…
The Text Uncanny Valley: Non-Monotonic Performance Degradation in LLM Information Retrieval
Zekai Tong, Ruiyao Xu, Aryan Shrivastava +2
Existing Large Language Model (LLM) benchmarks primarily focus on syntactically correct inputs, leaving a significant gap in evaluation on imperfect text. In this work, we study ho…
Iterative Finetuning is Mostly Idempotent
Zephaniah Roe, Jack Sanderson, Dang Nguyen +5
If a model has some behavioral tendency, such as sycophancy or misalignment, and it is trained on its own outputs, will the tendency be amplified in the next generation of models?…
Moral Mazes in the Era of LLMs
Dang Nguyen, Harvey Yiyun Fu, Peter West +2
Navigating complex social situations is an integral part of corporate life, ranging from giving critical feedback without hurting morale to rejecting requests without alienating te…
CLEAR: A Clinically-Grounded Tabular Framework for Radiology Report Evaluation
Yuyang Jiang, Chacha Chen, Shengyuan Wang +8
Existing metrics often lack the granularity and interpretability to capture nuanced clinical differences between candidate and ground-truth radiology reports, resulting in suboptim…
On the Effectiveness and Generalization of Race Representations for Debiasing High-Stakes Decisions
Dang Nguyen, Chenhao Tan
Understanding and mitigating biases is critical for the adoption of large language models (LLMs) in high-stakes decision-making. We introduce Admissions and Hiring, decision tasks…
Can Domain Experts Rely on AI Appropriately? A Case Study on AI-Assisted Prostate Cancer MRI Diagnosis
Chacha Chen, Han Liu, Jiamin Yang +11
Despite the growing interest in human-AI decision making, experimental studies with domain experts remain rare, largely due to the complexity of working with domain experts and the…
GPT-4V Cannot Generate Radiology Reports Yet
Yuyang Jiang, Chacha Chen, Dang Nguyen +2
GPT-4V's purported strong multimodal abilities raise interests in using it to automate radiology report writing, but there lacks thorough evaluations. In this work, we perform a sy…
OpenHEXAI: An Open-Source Framework for Human-Centered Evaluation of Explainable Machine Learning
Jiaqi Ma, Vivian Lai, Yiming Zhang +5
Recently, there has been a surge of explainable AI (XAI) methods driven by the need for understanding machine learning model behaviors in high-stakes scenarios. However, properly e…
Pragmatic Radiology Report Generation
Dang Nguyen, Chacha Chen, He He +1
When pneumonia is not found on a chest X-ray, should the report describe this negative observation or omit it? We argue that this question cannot be answered from the X-ray alone a…
Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation
Vivian Lai, Samuel Carton, Rajat Bhatnagar +3
Despite impressive performance in many benchmark datasets, AI models can still make mistakes, especially among out-of-distribution examples. It remains an open question how such im…