5 papers
A Course Correction in Steerability Evaluation: Revealing Miscalibration and Side Effects in LLMs
Trenton Chang, Tobias Schnabel, Adith Swaminathan +1
Despite advances in large language models (LLMs) on reasoning and instruction-following tasks, it is unclear whether they can reliably produce outputs aligned with a variety of use…
Measuring Model Performance in the Presence of an Intervention
Winston Chen, Michael W. Sjoding, Jenna Wiens
AI models are often evaluated based on their ability to predict the outcome of interest. However, in many AI for social impact applications, the presence of an intervention that af…
Conditional Front-door Adjustment for Heterogeneous Treatment Assignment Effect Estimation Under Non-adherence
Winston Chen, Trenton Chang, Jenna Wiens
Estimates of heterogeneous treatment assignment effects can inform treatment decisions. Under the presence of non-adherence (e.g., patients do not adhere to their assigned treatmen…
Estimating Misreporting in the Presence of Genuine Modification: A Causal Perspective
Dylan Zapzalka, Trenton Chang, Lindsay Warrenburg +5
In settings where ML models are used to inform the allocation of resources, agents affected by the allocation decisions might have an incentive to strategically change their featur…
Who's Gaming the System? A Causally-Motivated Approach for Detecting Strategic Adaptation
Trenton Chang, Lindsay Warrenburg, Sae-Hwan Park +3
In many settings, machine learning models may be used to inform decisions that impact individuals or entities who interact with the model. Such entities, or agents, may game model…