23 citations · 38 across the 15 of their papers we have counts for
12 papers · 1 filter
CultureMINE: Datasets and Methods for Improving the Cultural Capabilities of NLP Systems
Tania Chakraborty, Eylon Caplan, Zhaoqing Wu +4
In recent years, there has been a surge of interest in Cultural NLP, with substantial efforts to create globally inclusive NLP systems. The rapid growth of literature in this field…
T-FIX: Text-Based Explanations with Features Interpretable to eXperts
Shreya Havaldar, Weiqiu You, Chaehyeon Kim +12
As LLMs are deployed in knowledge-intensive settings (e.g., surgery, astronomy, therapy), users are often domain experts who expect not just answers, but explanations that mirror p…
Culturally-Aware Conversations: A Framework & Benchmark for LLMs
Shreya Havaldar, Sunny Rai, Young-Min Cho +1
Existing benchmarks that measure cultural adaptation in LLMs are misaligned with the actual challenges these models face when interacting with users from diverse cultural backgroun…
Towards Style Alignment in Cross-Cultural Translation
Shreya Havaldar, Adam Stein, Eric Wong +1
Successful communication depends on the speaker's intended style (i.e., what the speaker is trying to convey) aligning with the listener's interpreted style (i.e., what the listene…
Adaptively profiling models with task elicitation
Davis Brown, Prithvi Balehannina, Helen Jin +3
Language model evaluations often fail to characterize consequential failure modes, forcing experts to inspect outputs and build new benchmarks. We introduce task elicitation, a met…
Entailed Between the Lines: Incorporating Implication into NLI
Shreya Havaldar, Hamidreza Alvari, John Palowitch +3
Much of human communication depends on implication, conveying meaning beyond literal words to express a wider range of thoughts, intentions, and feelings. For models to better unde…
Building Knowledge-Guided Lexica to Model Cultural Variation
Shreya Havaldar, Salvatore Giorgi, Sunny Rai +4
Cultural variation exists between nations (e.g., the United States vs. China), but also within regions (e.g., California vs. Texas, Los Angeles vs. San Francisco). Measuring this r…
Comparing Styles across Languages: A Cross-Cultural Exploration of Politeness
Shreya Havaldar, Matthew Pressimone, Eric Wong +1
Understanding how styles differ across languages is advantageous for training both humans and computers to generate culturally appropriate text. We introduce an explanation framewo…
Multilingual Language Models are not Multicultural: A Case Study in Emotion
Shreya Havaldar, Sunny Rai, Bhumika Singhal +3
Emotions are experienced and expressed differently across the world. In order to use Large Language Models (LMs) for multilingual tasks that require emotional sensitivity, LMs must…
TopEx: Topic-based Explanations for Model Comparison
Shreya Havaldar, Adam Stein, Eric Wong +1
Meaningfully comparing language models is challenging with current explanation methods. Current explanations are overwhelming for humans due to large vocabularies or incomparable a…
Psychological Metrics for Dialog System Evaluation
Salvatore Giorgi, Shreya Havaldar, Farhan Ahmed +6
We present metrics for evaluating dialog systems through a psychologically-grounded "human" lens in which conversational agents express a diversity of both states (e.g., emotion) a…
Faithful Chain-of-Thought Reasoning
Qing Lyu, Shreya Havaldar, Adam Stein +5
While Chain-of-Thought (CoT) prompting boosts Language Models' (LM) performance on a gamut of complex reasoning tasks, the generated reasoning chain does not necessarily reflect ho…