565 citations · 989 across the 82 of their papers we have counts for
32 papers · 1 filter
Social Caption: Evaluating Social Understanding in Multimodal Models
Leena Mathur, Bhaavanaa Thumu, Youssouf Kebe +1
Social understanding abilities are crucial for multimodal large language models (MLLMs) to interpret human social interactions. We introduce SOCIAL CAPTION, a framework grounded in…
Aligning Dialogue Agents with Global Feedback via Large Language Model Multimodal Reward Decomposition
Dong Won Lee, Hae Won Park, Cynthia Breazeal +1
We propose a large language model based reward decomposition framework for aligning dialogue agents using only a single session-level feedback signal. We leverage the reasoning cap…
Social Genome: Grounded Social Reasoning Abilities of Multimodal Models
Leena Mathur, Marian Qian, Paul Pu Liang +1
Social reasoning abilities are crucial for AI systems to effectively interpret and respond to multimodal human communication and interaction within social contexts. We introduce SO…
Towards Implicit Bias Detection and Mitigation in Multi-Agent LLM Interactions
Angana Borah, Rada Mihalcea
As Large Language Models (LLMs) continue to evolve, they are increasingly being employed in numerous studies to simulate societies and execute diverse social tasks. However, LLMs a…
Isolated Causal Effects of Natural Language
Victoria Lin, Louis-Philippe Morency, Eli Ben-Michael
As language technologies become widespread, it is important to understand how changes in language affect reader perceptions and behaviors. These relationships may be formalized as…
Improving Dialogue Agents by Decomposing One Global Explicit Annotation with Local Implicit Multimodal Feedback
Dong Won Lee, Hae Won Park, Yoon Kim +2
We describe an approach for aligning an LLM-based dialogue agent based on global (i.e., dialogue-level) rewards, while also taking into account naturally-occurring multimodal signa…