10 papers
T-FIX: Text-Based Explanations with Features Interpretable to eXperts
Shreya Havaldar, Weiqiu You, Chaehyeon Kim +12
As LLMs are deployed in knowledge-intensive settings (e.g., surgery, astronomy, therapy), users are often domain experts who expect not just answers, but explanations that mirror p…
Culturally-Aware Conversations: A Framework & Benchmark for LLMs
Shreya Havaldar, Sunny Rai, Young-Min Cho +1
Existing benchmarks that measure cultural adaptation in LLMs are misaligned with the actual challenges these models face when interacting with users from diverse cultural backgroun…
Probabilistic Soundness Guarantees in LLM Reasoning Chains
Weiqiu You, Anton Xue, Shreya Havaldar +4
In reasoning chains generated by large language models (LLMs), initial errors often propagate and undermine the reliability of the final conclusion. Current LLM-based error detecti…
Adaptively profiling models with task elicitation
Davis Brown, Prithvi Balehannina, Helen Jin +3
Language model evaluations often fail to characterize consequential failure modes, forcing experts to inspect outputs and build new benchmarks. We introduce task elicitation, a met…
The FIX Benchmark: Extracting Features Interpretable to eXperts
Helen Jin, Shreya Havaldar, Chaehyeon Kim +10
Feature-based methods are commonly used to explain model predictions, but these methods often implicitly assume that interpretable features are readily available. However, this is…
Towards Style Alignment in Cross-Cultural Translation
Shreya Havaldar, Adam Stein, Eric Wong +1
Successful communication depends on the speaker's intended style (i.e., what the speaker is trying to convey) aligning with the listener's interpreted style (i.e., what the listene…