Implicit Humanization in Everyday LLM Moral Judgments

Abstract

Recent adoption of conversational information systems has expanded the scope of user queries to include complex tasks such as personal advice-seeking. However, we identify a specific type of sought advice-a request for a moral judgment (i.e. "who was wrong?") in a social conflict-as an implicitly humanizing query which carries potentially harmful anthropomorphic projections. In this study, we examine the reinforcement of these assumptions in the responses of four major general-purpose LLMs through the use of linguistic, behavioral, and cognitive anthropomorphic cues. We also contribute a novel dataset of simulated user queries for moral judgments. We find current LLM system responses reinforce implicit humanization in queries, potentially exacerbating risks like overreliance or misplaced trust. We call for future work to expand the understanding of anthropomorphism to include implicit userside humanization and to design solutions that address user needs while correcting misaligned expectations of model capabilities.

6 pages, 3 figures, Published in CHIIR '26

References in corpus (13)

• The Fallacy of AI Functionality
• What Large Language Models Know and What People Think They Know
• Self-Instruct: Aligning Language Models with Self-Generated Instructions
• Leveraging Large Language Models for Patient Engagement: The Power of Conversational AI in Digital Health
• Lawyer LLaMA Technical Report
• Investigating Affective Use and Emotional Well-being on ChatGPT
• Objection Overruled! Lay People can Distinguish Large Language Models from Lawyers, but still Favour Advice from an LLM
• The 'Problem' of Human Label Variation: On Ground Truth in Data, Modeling and Evaluation
• Mirages: On Anthropomorphism in Dialogue Systems
• Rel-A.I.: An Interaction-Centered Approach To Measuring Human-LM Reliance
• The Art of Saying No: Contextual Noncompliance in Language Models
• HumT DumT: Measuring and controlling human-like language in LLMs
• The Constant in HATE: Analyzing Toxicity in Reddit across Topics and Languages