7 citations · 25 across the 38 of their papers we have counts for
39 papers
CultureConverse: A Multilingual Multi-turn Simulation Harness for Culturally Grounded Assistance in East and Southeast Asia
Bryan Chen Zhengyu Tan, Weihua Zheng, Thong T. Doan +30
Current cultural evaluations for large language models (LLMs) often reduce culture to single-turn factual recall via MCQs, failing to capture a common use case: users seeking pract…
AI Evaluation Should Require Standardized Item-Level Data Releases
Han Jiang, Susu Zhang, Dongyao Zhu +6
This position paper argues that standardized item-level benchmark data should become the default infrastructure for AI evaluation. Current evaluations suffer from underspecified it…
Can Persona-Prompted LLMs Emulate Subgroup Values? An Empirical Analysis of Generalisability and Fairness in Cultural Alignment
Bryan Chen Zhengyu Tan, Zhengyuan Liu, Xiaoyuan Yi +4
Despite their global prevalence, many Large Language Models (LLMs) are aligned to a monolithic, often Western-centric set of values. This paper investigates the more challenging ta…
Human Values Matter: Investigating How Misalignment Shapes Collective Behaviors in LLM Agent Communities
Xiangxu Zhang, Jiamin Wang, Qinlin Zhao +6
As LLMs become increasingly integrated into human society, evaluating their orientations on human values from social science has drawn growing attention. Nevertheless, it is still…
Distributional Open-Ended Evaluation of LLM Cultural Value Alignment Based on Value Codebook
Jaehyeok Lee, Xiaoyuan Yi, Jing Yao +4
As LLMs are globally deployed, aligning their cultural value orientations is critical for safety and user engagement. However, existing benchmarks face the Construct-Composition-Co…
Does LLM Alignment Really Need Diversity? An Empirical Study of Adapting RLVR Methods for Moral Reasoning
Zhaowei Zhang, Xiaohan Liu, Xuekai Zhu +6
Reinforcement learning with verifiable rewards (RLVR) has achieved remarkable success in logical reasoning tasks, yet whether large language model (LLM) alignment requires fundamen…
Contextualized Privacy Defense for LLM Agents
Yule Wen, Yanzhe Zhang, Jianxun Lian +3
LLM agents increasingly act on users' personal information, yet existing privacy defenses remain limited in both design and adaptability. Most prior approaches rely on static or pa…
On the Dynamics of Multi-Agent LLM Communities Driven by Value Diversity
Muhua Huang, Qinlin Zhao, Xiaoyuan Yi +1
As Large Language Models (LLM) based multi-agent systems become increasingly prevalent, the collective behaviors, e.g., collective intelligence, of such artificial communities have…
Knowing Your Uncertainty -- On the application of LLM in social sciences
Bolun Zhang, Linzhuo Li, Yunqi Chen +4
Large language models (LLMs) are rapidly being integrated into computational social science research, yet their blackboxed training and designed stochastic elements in inference po…
MMA-ASIA: A Multilingual and Multimodal Alignment Framework for Culturally-Grounded Evaluation
Weihua Zheng, Zhengyuan Liu, Tanmoy Chakraborty +32
Large language models (LLMs) are now used worldwide, yet their multimodal understanding and reasoning often degrade outside Western, high-resource settings. We propose MMA-ASIA, a…
Counterfactual Reasoning for Steerable Pluralistic Value Alignment of Large Language Models
Hanze Guo, Jing Yao, Xiao Zhou +2
As large language models (LLMs) become increasingly integrated into applications serving users across diverse cultures, communities and demographics, it is critical to align LLMs w…
MoVa: Towards Generalizable Classification of Human Morals and Values
Ziyu Chen, Junfei Sun, Chenxi Li +6
Identifying human morals and values embedded in language is essential to empirical studies of communication. However, researchers often face substantial difficulty navigating the d…
The Morality of Probability: How Implicit Moral Biases in LLMs May Shape the Future of Human-AI Symbiosis
Eoin O'Doherty, Nicole Weinrauch, Andrew Talone +4
Artificial intelligence (AI) is advancing at a pace that raises urgent questions about how to align machine decision-making with human moral values. This working paper investigates…
IROTE: Human-like Traits Elicitation of Large Language Model via In-Context Self-Reflective Optimization
Yuzhuo Bai, Shitong Duan, Muhua Huang +7
Trained on various human-authored corpora, Large Language Models (LLMs) have demonstrated a certain capability of reflecting specific human-like traits (e.g., personality or values…
The Incomplete Bridge: How AI Research (Mis)Engages with Psychology
Han Jiang, Pengda Wang, Xiaoyuan Yi +2
Social sciences have accumulated a rich body of theories and methodologies for investigating the human mind and behaviors, while offering valuable insights into the design and unde…
MoHoBench: Assessing Honesty of Multimodal Large Language Models via Unanswerable Visual Questions
Yanxu Zhu, Shitong Duan, Xiangxu Zhang +7
Recently Multimodal Large Language Models (MLLMs) have achieved considerable advancements in vision-language tasks, yet produce potentially harmful or untrustworthy content. Despit…
PICACO: Pluralistic In-Context Value Alignment of LLMs via Total Correlation Optimization
Han Jiang, Dongyao Zhu, Xiaoyuan Yi +3
In-Context Learning has shown great potential for aligning Large Language Models (LLMs) with human values, helping reduce harmful outputs and accommodate diverse preferences withou…
MotiveBench: How Far Are We From Human-Like Motivational Reasoning in Large Language Models?
Xixian Yong, Jianxun Lian, Xiaoyuan Yi +2
Large language models (LLMs) have been widely adopted as the core of agent frameworks in various scenarios, such as social simulations and AI companions. However, the extent to whi…
Unintended Harms of Value-Aligned LLMs: Psychological and Empirical Insights
Sooyung Choi, Jaehyeok Lee, Xiaoyuan Yi +3
The application scope of Large Language Models (LLMs) continues to expand, leading to increasing interest in personalized LLMs that align with human values. However, aligning these…
AdAEM: An Adaptively and Automated Extensible Measurement of LLMs' Value Difference
Jing Yao, Shitong Duan, Xiaoyuan Yi +6
Assessing Large Language Models'(LLMs) underlying value differences enables comprehensive comparison of their misalignment, cultural adaptability, and biases. Nevertheless, current…
CAReDiO: Cultural Alignment via Representativeness and Distinctiveness Guided Data Optimization
Jing Yao, Xiaoyuan Yi, Jindong Wang +2
As Large Language Models (LLMs) are deployed across diverse regions, aligning them with pluralistic cultures is crucial for improving user engagement and mitigating cultural confli…
Beyond BFI: The CSI for Enhanced Reliability and Validity in Evaluating LLM Personality Traits
Huanhuan Ma, Haisong Gong, Xiaoyuan Yi +3
As large language models (LLMs) increasingly function as human-like assistants exhibiting human-like personality traits, understanding their behavioral characteristics becomes esse…
Research Superalignment Should Advance Now with Alternating Competence and Conformity Optimization
HyunJin Kim, Xiaoyuan Yi, Jing Yao +4
The recent leap in AI capabilities, driven by big generative models, has sparked the possibility of achieving Artificial General Intelligence (AGI) and further triggered discussion…
Value Compass Benchmarks: A Platform for Fundamental and Validated Evaluation of LLMs Values
Jing Yao, Xiaoyuan Yi, Shitong Duan +8
As Large Language Models (LLMs) achieve remarkable breakthroughs, aligning their values with humans has become imperative for their responsible development and customized applicati…