30 citations · 40 across the 16 of their papers we have counts for
18 papers
XYBench: Can LLMs Respond Pragmatically to Queries with Misconceptions?
Akhila Yerukola, Jena D. Hwang, Mingqian Zheng +5
When non-expert users ask LLMs for assistance, their queries can often have misconceptions (e.g., "How do I parse XML with regex?"). In such cases, often referred to as the XY-prob…
Social Gym and SPaRTan: Benchmarking and Improving LLM Social Reasoning via Multi-Agent Game Tournaments
Keyu He, Xuhui Zhou, Maarten Sap
LLM agents are increasingly deployed in multi-agent social settings where they must cooperate, negotiate, and adapt to other agents. Measuring and improving these social skills is…
Ouvia: A User-centered Framework for Measuring Usability of Speech Translation in Real-World Communication Scenarios
Giuseppe Attanasio, Beatrice Savoldi, Daniel Chechelnitsky +4
Speech translation (ST) is increasingly adopted in user applications, yet its evaluation largely focuses on decontextualized testbeds and holistic quality, rather than end users' c…
OdysSim: Building Foundation Models for Human Behavior Simulation
Xuhui Zhou, Weiwei Sun, Weihua Du +6
Large language models are increasingly deployed as human simulators for interactive evaluation and social simulation. Yet helpfulness-driven post-training pulls them toward a homog…
Useless but Safe? Benchmarking Utility Recovery with User Intent Clarification in Multi-Turn Conversations
Mingqian Zheng, Malia Morgan, Liwei Jiang +2
Current LLM safety alignment techniques improve model robustness against adversarial attacks, but overlook whether and how LLMs can recover helpfulness when benign users clarify th…
GoodPoint: Learning Constructive Scientific Paper Feedback from Author Responses
Jimin Mun, Chani Jung, Xuhui Zhou +2
While LLMs hold significant potential to transform scientific research, we advocate for their use to augment and empower researchers rather than to automate research without human…
Locating Translation as a Craft in the Age of AI
Daniel Chechelnitsky, Sireesh Gururaja, Seyi Olojo +4
Rapid development of Large Language Models (LLMs) and similar automated approaches for translation tasks is increasingly affecting the landscape of translation technologies. As con…
Agents of Chaos
Natalie Shapira, Chris Wendler, Avery Yen +35
We report an exploratory red-teaming study of autonomous language-model-powered agents deployed in a live laboratory environment with persistent memory, email accounts, Discord acc…
Training Proactive and Personalized LLM Agents
Weiwei Sun, Xuhui Zhou, Weihua Du +5
Despite rapid progress, current AI agents are primarily optimized for isolated task completion. We argue for a paradigm shift toward training agents as collaborators that communica…
TOM-SWE: User Mental Modeling For Software Engineering Agents
Xuhui Zhou, Valerie Chen, Zora Zhiruo Wang +3
Recent advances in coding agents have made them capable of planning, editing, running, and testing complex code bases. Despite their growing ability in coding tasks, these systems…
The PIMMUR Principles: Ensuring Validity in Collective Behavior of LLM Societies
Jiaxu Zhou, Jen-tse Huang, Xuhui Zhou +5
Large language models (LLMs) are increasingly used to simulate human collective behavior, yet claims that such simulations are human-like remain largely untested. We conducted a sy…
Social World Models
Xuhui Zhou, Jiarui Liu, Akhila Yerukola +2
Humans intuitively navigate social interactions by simulating unspoken dynamics and reasoning about others' perspectives, even with limited information. In contrast, AI systems str…
1-2-3 Check: Enhancing Contextual Privacy in LLM via Multi-Agent Reasoning
Wenkai Li, Liwen Sun, Zhenxiang Guan +2
Addressing contextual privacy concerns remains challenging in interactive settings where large language models (LLMs) process information from multiple sources (e.g., summarizing m…
OpenAgentSafety: A Comprehensive Framework for Evaluating Real-World AI Agent Safety
Sanidhya Vijayvargiya, Aditya Bharat Soni, Xuhui Zhou +4
Recent advances in AI agents capable of solving complex, everyday tasks, from scheduling to customer service, have enabled deployment in real-world settings, but their possibilitie…
SoMi-ToM: Evaluating Multi-Perspective Theory of Mind in Embodied Social Interactions
Xianzhe Fan, Xuhui Zhou, Chuanyang Jin +3
Humans continuously infer the states, goals, and behaviors of others by perceiving their surroundings in dynamic, real-world social interactions. However, most Theory of Mind (ToM)…
Words Like Knives: Backstory-Personalized Modeling and Detection of Violent Communication
Jocelyn Shen, Akhila Yerukola, Xuhui Zhou +3
Conversational breakdowns in close relationships are deeply shaped by personal histories and emotional context, yet most NLP research treats conflict detection as a general task, o…
Breaking mBad! Supervised Fine-tuning for Cross-Lingual Detoxification
Himanshu Beniwal, Youngwoo Kim, Maarten Sap +2
As large language models (LLMs) become increasingly prevalent in global applications, ensuring that they are toxicity-free across diverse linguistic contexts remains a critical cha…
Medical Hallucinations in Foundation Models and Their Impact on Healthcare
Yubin Kim, Hyewon Jeong, Shan Chen +24
Hallucinations in foundation models arise from autoregressive training objectives that prioritize token-likelihood optimization over epistemic accuracy, fostering overconfidence an…