13 papers
IdeaScientist: Orchestrating Agents for Grounded Scientific Ideation
Jiarui Liu, Renjie Tao, Yiwei Liao +15
Despite rapid progress in automating scientific research, generating promising and well grounded research solutions remains a central challenge. We isolate research ideation as a s…
On the Behavioral Traits of LLM Agents
Haokai Zhao, Jie Gao, Yunze Xiao +5
Users increasingly describe different AI agents as distinct colleagues to work with. AI personality research aims to quantify such impressions by attributing human- like "traits" t…
Harbor Adapters and Harbor-Index: Infrastructure and a Curated Meta-Dataset for Large-Scale Agentic Evaluation
Lin Shi, Haowei Lin, Zixuan Zhu +123
Evaluating agents on the growing number of agentic benchmarks is challenging because they often require complex environments and agent integrations. We introduce Harbor Adapters, a…
Group Alignment-Induced Sycophancy: A Two-Sided Evaluation of Steerable Pluralistic Alignment
Haokai Zhao, Yunze Xiao, Weihao Xuan +3
Group alignment adapts a language model to a demographic group to produce responses that reflect the group's opinions, values, and preferences. Sycophancy, a well-documented by-pro…
ExpressionCueLens: A Cross-Cultural Analysis of Human-AI Companion Conversations on Social Media
Lynnette Hui Xian Ng, Yunze Xiao, Lionel Z. Wang +2
LLM-based AI companion agents are increasingly being perceived not only as tools but also as social companions. On social media, people recount conversations where these agents com…
MixSD: Mixed Contextual Self-Distillation for Knowledge Injection
Jiarui Liu, Lechen Zhang, Yongjin Yang +5
Supervised fine-tuning (SFT) is widely used to inject new knowledge into language models, but it often degrades pretrained capabilities such as reasoning and general-domain perform…
Knowledge Index of Noah's Ark
Sheng Jin, Minghao Liu, Yunze Xiao +24
Knowledge benchmarks for LLMs face three issues: scaling-driven designs that do not operationalize disciplinary representativeness; flat-payment annotation that permits lazy consen…
Is LLM's Moral Decision Compositional?
Weijia Zhang, Ruiqi Chen, Yunze Xiao +1
Moral evaluations of language models typically infer a model's values from its choices between single acts, yet in everyday life a person is judged on their history of good and bad…
The Chameleon's Limit: Investigating Persona Collapse and Homogenization in Large Language Models
Yunze Xiao, Vivienne J. Zhang, Chenghao Yang +3
Large language models (LLMs) have powered multi-agent simulations such as survey respondents or application users. However, LLMs lack the population-level diversity required by the…
Say Something Else: Rethinking Contextual Privacy as Information Sufficiency
Yunze Xiao, Wenkai Li, Xiaoyuan Wu +3
LLM agents increasingly draft messages on behalf of users, yet users routinely overshare sensitive information and disagree on what counts as private. Existing systems support only…
Towards Valid Student Simulation with Large Language Models
Zhihao Yuan, Yunze Xiao, Ming Li +4
This paper presents a conceptual and methodological framework for large language model (LLM) based student simulation in educational settings. The authors identify a core failure m…
Sentipolis: Emotion-Aware Agents for Social Simulations
Chiyuan Fu, Lyuhao Chen, Yunze Xiao +3
LLM agents are increasingly used for social simulation, yet emotion is often treated as a transient cue, causing emotional amnesia and weak long-horizon continuity. We present Sent…
Taming Object Hallucinations with Verified Atomic Confidence Estimation
Jiarui Liu, Weihao Xuan, Zhijing Jin +1
Multimodal Large Language Models (MLLMs) often suffer from hallucinations, particularly errors in object existence, attributes, or relations, which undermine their reliability. We…