9 papers
Language Scent: Exploring Cross-Language Information Navigation
Jiawen Stefanie Zhu, Katharina Reinecke, Tanushree Mitra
While multilingual users often switch between languages when seeking information, this process remains undersupported by current systems where information is typically siloed by la…
ValueCompass: A Framework for Measuring Contextual Value Alignment Between Human and LLMs
Hua Shen, Tiffany Knearem, Reshmi Ghosh +4
As AI systems become more advanced, ensuring their alignment with a diverse range of individuals and societal values becomes increasingly critical. But how can we capture fundament…
Who's Asking? Simulating Role-Based Questions for Conversational AI Evaluation
Navreet Kaur, Hoda Ayad, Hayoung Jung +3
Language model users often embed personal and social context in their questions. The asker's role -- implicit in how the question is framed -- creates specific needs for an appropr…
ABLEIST: Intersectional Disability Bias in LLM-Generated Hiring Scenarios
Mahika Phutane, Hayoung Jung, Matthew Kim +2
Large language models (LLMs) are increasingly under scrutiny for perpetuating identity-based discrimination in high-stakes domains such as hiring, particularly against people with…
Mind the Value-Action Gap: Do LLMs Act in Alignment with Their Values?
Hua Shen, Nicholas Clark, Tanushree Mitra
Existing research primarily evaluates the values of LLMs by examining their stated inclinations towards specific values. However, the "Value-Action Gap," a phenomenon rooted in env…
MythTriage: Scalable Detection of Opioid Use Disorder Myths on a Video-Sharing Platform
Hayoung Jung, Shravika Mittal, Ananya Aatreya +3
Understanding the prevalence of misinformation in health topics online can inform public health policies and interventions. However, measuring such misinformation at scale remains…