4 citations · 7 across the 8 of their papers we have counts for
9 papers
Learning to Simulate Individuals from Macro Social Signals
Yining Zhao, Bushi Liu, Haofei Yu +5
Large language models are increasingly used to simulate how individuals respond to new situations, yet the behavioral reasoning behind these responses is either inherited from pret…
OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks
Mengqi Yuan, Zilong Zhou, Xinzhuang Xiong +33
Existing computer-use benchmarks fail to capture the realism, complexity, and long-horizon demands of real-world computer use, limiting their ability to reveal the limitations of f…
The Hitchhikers Guide to Rubric Quality Understanding and Enrichment
Zhengyang Qi, Ankit Aich, Charles Dickens +7
Rubrics distill notions of expert quality and measure agent performance. However, the quality of rubrics themselves have not been systematically measured and are often left to down…
Automating Benchmark Design
Amanda Dsouza, Harit Vishwakarma, Zhengyang Qi +6
The rapid progress and widespread deployment of LLMs and LLM-powered agents has outpaced our ability to evaluate them. Hand-crafted, static benchmarks are the primary tool for asse…
Sotopia-RL: Reward Design for Social Intelligence
Haofei Yu, Zhengyang Qi, Yining Zhao +6
Social intelligence has become a critical capability for large language models (LLMs), enabling them to engage effectively in real-world social tasks such as collaboration and nego…
SOTOPIA-: Interactive Learning of Socially Intelligent Language Agents
Ruiyi Wang, Haofei Yu, Wenxin Zhang +5
Humans learn social skills through both imitation and social interaction. This social learning process is largely understudied by existing research on building language agents. Mot…
Long-Horizon Dialogue Understanding for Role Identification in the Game of Avalon with Large Language Models
Simon Stepputtis, Joseph Campbell, Yaqi Xie +6
Deception and persuasion play a critical role in long-horizon dialogues between multiple parties, especially when the interests, goals, and motivations of the participants are not…
MMoE: Enhancing Multimodal Models with Mixtures of Multimodal Interaction Experts
Haofei Yu, Zhengyang Qi, Lawrence Jang +3
Advances in multimodal models have greatly improved how interactions relevant to various tasks are modeled. Today's multimodal models mainly focus on the correspondence between ima…
SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents
Xuhui Zhou, Hao Zhu, Leena Mathur +8
Humans are social beings; we pursue social goals in our daily interactions, which is a crucial aspect of social intelligence. Yet, AI systems' abilities in this realm remain elusiv…