7 citations · 14 across the 25 of their papers we have counts for
27 papers
CyberArena: Benchmarking AI Agents in Live Symmetric Attack-and-Defense Combat
Bangya Liu, Jianhong Tu, Yueyang Pan +18
AI agents are demonstrating increasingly strong offensive and defensive cybersecurity capabilities in the real world. Current benchmarks mostly evaluate agents in isolation against…
On the Behavioral Traits of LLM Agents
Haokai Zhao, Jie Gao, Yunze Xiao +5
Users increasingly describe different AI agents as distinct colleagues to work with. AI personality research aims to quantify such impressions by attributing human- like "traits" t…
MicroVerse: An Instrument for Measuring Self-Authored Identity Drift in Long-Horizon Multi-Agent Language-Model Simulations
Sky Ng, Brihi Joshi, Ishan Gupta +47
Long-horizon, multi-agent language model (LM) simulations are widely proposed for studying social behavior, yet instruments to measure whether persona-conditioned agents maintain i…
PersonaEval: Persona-Based User Simulation for Evaluating Interactive Applications
Yifan Simon Liu, Qianfeng Wen, Yilan Fan +40
Real user studies are important for understanding how people interact with systems under test or already deployed. In practice, however, they are often costly, time-consuming, and…
Group Alignment-Induced Sycophancy: A Two-Sided Evaluation of Steerable Pluralistic Alignment
Haokai Zhao, Yunze Xiao, Weihao Xuan +3
Group alignment adapts a language model to a demographic group to produce responses that reflect the group's opinions, values, and preferences. Sycophancy, a well-documented by-pro…
ExpressionCueLens: A Cross-Cultural Analysis of Human-AI Companion Conversations on Social Media
Lynnette Hui Xian Ng, Yunze Xiao, Lionel Z. Wang +2
LLM-based AI companion agents are increasingly being perceived not only as tools but also as social companions. On social media, people recount conversations where these agents com…
PACE: A Proxy for Agentic Capability Evaluation
Yueqi Song, Lintang Sutawika, Jiarui Liu +8
Evaluating LLM agents on benchmarks like SWE-Bench and GAIA can be expensive, time-consuming, and requires complex infrastructure. A single evaluation can cost thousands of dollars…
Knowledge Index of Noah's Ark
Sheng Jin, Minghao Liu, Yunze Xiao +24
Knowledge benchmarks for LLMs face three issues: scaling-driven designs that do not operationalize disciplinary representativeness; flat-payment annotation that permits lazy consen…
Dr. DocBench: A Comprehensive Benchmark for Expert-Level and Difficult Document Parsing
Minglai Yang, Xinyan Velocity Yu, Pengyuan Li +22
Document parsing and recognition are fundamental capabilities for vision-language models (VLMs) and document processing systems. However, existing Optical Character Recognition (OC…
Is LLM's Moral Decision Compositional?
Weijia Zhang, Ruiqi Chen, Yunze Xiao +1
Moral evaluations of language models typically infer a model's values from its choices between single acts, yet in everyday life a person is judged on their history of good and bad…
The Chameleon's Limit: Investigating Persona Collapse and Homogenization in Large Language Models
Yunze Xiao, Vivienne J. Zhang, Chenghao Yang +3
Large language models (LLMs) have powered multi-agent simulations such as survey respondents or application users. However, LLMs lack the population-level diversity required by the…
Superminds Test: Actively Evaluating Collective Intelligence of Agent Society via Probing Agents
Xirui Li, Ming Li, Yunze Xiao +4
Collective intelligence refers to the ability of a group to achieve outcomes beyond what any individual member can accomplish alone. As large language model agents scale to populat…
Say Something Else: Rethinking Contextual Privacy as Information Sufficiency
Yunze Xiao, Wenkai Li, Xiaoyuan Wu +3
LLM agents increasingly draft messages on behalf of users, yet users routinely overshare sensitive information and disagree on what counts as private. Existing systems support only…
Towards Valid Student Simulation with Large Language Models
Zhihao Yuan, Yunze Xiao, Ming Li +4
This paper presents a conceptual and methodological framework for large language model (LLM) based student simulation in educational settings. The authors identify a core failure m…
Sentipolis: Emotion-Aware Agents for Social Simulations
Chiyuan Fu, Lyuhao Chen, Yunze Xiao +3
LLM agents are increasingly used for social simulation, yet emotion is often treated as a transient cue, causing emotional amnesia and weak long-horizon continuity. We present Sent…
Hire Your Anthropologist! Rethinking Culture Benchmarks Through an Anthropological Lens
Mai AlKhamissi, Yunze Xiao, Badr AlKhamissi +1
Cultural evaluation of large language models has become increasingly important, yet current benchmarks often reduce culture to static facts or homogeneous values. This view conflic…
Humanizing Machines: Rethinking LLM Anthropomorphism Through a Multi-Level Framework of Design
Yunze Xiao, Lynnette Hui Xian Ng, Jiarui Liu +1
Large Language Models (LLMs) increasingly exhibit \textbf{anthropomorphism} characteristics -- human-like qualities portrayed across their outlook, language, behavior, and reasonin…
Synthetic Socratic Debates: Examining Persona Effects on Moral Decision and Persuasion Dynamics
Jiarui Liu, Yueqi Song, Yunze Xiao +5
As large language models (LLMs) are increasingly used in morally sensitive domains, it is crucial to understand how persona traits affect their moral reasoning and persuasive behav…
Embracing Contradiction: Theoretical Inconsistency Will Not Impede the Road of Building Responsible AI Systems
Gordon Dai, Yunze Xiao
This position paper argues that the theoretical inconsistency often observed among Responsible AI (RAI) metrics, such as differing fairness definitions or tradeoffs between accurac…
Understanding Aha Moments: from External Observations to Internal Mechanisms
Shu Yang, Junchao Wu, Xin Chen +4
Large Reasoning Models (LRMs), capable of reasoning through complex problems, have become crucial for tasks like programming, mathematics, and commonsense reasoning. However, a key…
MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluation
Weihao Xuan, Rui Yang, Heli Qi +29
Existing large language model (LLM) evaluation benchmarks primarily focus on English, while current multilingual tasks lack parallel questions that specifically assess cross-lingui…
ToxiCloakCN: Evaluating Robustness of Offensive Language Detection in Chinese with Cloaking Perturbations
Yunze Xiao, Yujia Hu, Kenny Tsu Wei Choo +1
Detecting hate speech and offensive language is essential for maintaining a safe and respectful digital environment. This study examines the limitations of state-of-the-art large l…
Surveying Attitudinal Alignment Between Large Language Models Vs. Humans Towards 17 Sustainable Development Goals
Qingyang Wu, Ying Xu, Tingsong Xiao +8
Large Language Models (LLMs) have emerged as potent tools for advancing the United Nations' Sustainable Development Goals (SDGs). However, the attitudinal disparities between LLMs…
Chinese Offensive Language Detection:Current Status and Future Directions
Yunze Xiao, Houda Bouamor, Wajdi Zaghouani
Despite the considerable efforts being made to monitor and regulate user-generated content on social media platforms, the pervasiveness of offensive language, such as hate speech o…