4 citations · 5 across the 10 of their papers we have counts for
12 papers
Retrieval Sensitivity to Identity Signals in Queries
Andrew Tang, Nicholas Deas, Kathleen McKeown +1
Dense retrievers decide which documents reach users and the language models that use them, yet they are typically evaluated with neutral queries. We ask whether the identity signal…
Empath: Tracing Multi-Level Emotion Dynamics in Crisis Counseling Dialogues
Ziwei Gong, Yuchen Huang, Wen Liang +4
Emotion dynamics are critical for understanding crisis-support conversations, yet most computational work treats emotion as static utterance-level labels. We introduce EMPATH, a fr…
Whose Story Gets Told? Positionality and Bias in LLM Summaries of Life Narratives
Melanie Subbiah, Haaris Mian, Nicholas Deas +3
Increasingly, studies are exploring using Large Language Models (LLMs) for accelerated or scaled qualitative analysis of text data. While we can compare LLM accuracy against human…
Artificial Impressions: Evaluating Large Language Model Behavior Through the Lens of Trait Impressions
Nicholas Deas, Kathleen McKeown
We introduce and study artificial impressions--patterns in LLMs' internal representations of prompts that resemble human impressions and stereotypes based on language. We fit linea…
Reranking-based Generation for Unbiased Perspective Summarization
Narutatsu Ri, Nicholas Deas, Kathleen McKeown
Generating unbiased summaries in real-world settings such as political perspective summarization remains a crucial application of Large Language Models (LLMs). Yet, existing evalua…
AdvSumm: Adversarial Training for Bias Mitigation in Text Summarization
Mukur Gupta, Nikhil Reddy Varimalla, Nicholas Deas +2
Large Language Models (LLMs) have achieved impressive performance in text summarization and are increasingly deployed in real-world applications. However, these systems often inher…
Counterfactual Simulatability of LLM Explanations for Generation Tasks
Marvin Limpijankit, Yanda Chen, Melanie Subbiah +2
LLMs can be unpredictable, as even slight alterations to the prompt can cause the output to change in unexpected ways. Thus, the ability of models to accurately explain their behav…
Data Caricatures: On the Representation of African American Language in Pretraining Corpora
Nicholas Deas, Blake Vente, Amith Ananthram +5
With a combination of quantitative experiments, human judgments, and qualitative analyses, we evaluate the quantity and quality of African American Language (AAL) representation in…
Rejected Dialects: Biases Against African American Language in Reward Models
Joel Mire, Zubin Trivadi Aysola, Daniel Chechelnitsky +3
Preference alignment via reward models helps build safe, helpful, and reliable large language models (LLMs). However, subjectivity in preference judgments and the lack of represent…
Summarization of Opinionated Political Documents with Varied Perspectives
Nicholas Deas, Kathleen McKeown
Global partisan hostility and polarization has increased, and this polarization is heightened around presidential elections. Models capable of generating accurate summaries of dive…
MASIVE: Open-Ended Affective State Identification in English and Spanish
Nicholas Deas, Elsbeth Turcan, Iván Pérez Mejía +1
In the field of emotion analysis, much NLP research focuses on identifying a limited number of discrete emotion categories, often applied across languages. These basic sets, howeve…
Evaluation of African American Language Bias in Natural Language Generation
Nicholas Deas, Jessi Grieser, Shana Kleiner +3
We evaluate how well LLMs understand African American Language (AAL) in comparison to their performance on White Mainstream English (WME), the encouraged "standard" form of English…