10 papers
Representational Control over Self-Report & Behavior Coherence in LLM Risk-Taking
Rafal Kocielnik, Peiyang Song, Pengrui Han +4
Self-report is an appealing low-cost probe of an LLM's dispositions, but recent work finds only selective agreement between what models report and how they behave. Prior accounts e…
Rethinking Psychometric Evaluation of LLMs: When and Why Self-Reports Predict Behavior
Rafal Kocielnik, Pengrui Han, Peiyang Song +5
Anticipating LLM behavioral tendencies from low-cost psychometric probes is critical for safe deployment, but only if self-reports (SR) reliably predict behavior. Recent work docum…
A Multi-Agent LLM Framework for Rating the Quality of Surgical Feedback
Rafal Kocielnik, J. Everett Knudsen, Steven Y. Cen +7
Verbal feedback delivered by attending surgeons in the operating room plays a critical formative role in resident trainee skill acquisition. Yet, assessing the quality of trainer f…
Generating Natural-Language Surgical Feedback: From Structured Representation to Domain-Grounded Evaluation
Firdavs Nasriddinov, Rafal Kocielnik, Anima Anandkumar +1
High-quality intraoperative feedback from a surgical trainer is pivotal for improving trainee performance and long-term skill acquisition. Automating natural, trainer-style feedbac…
Beyond Ethics: How Inclusive Innovation Drives Economic Returns in Medical AI
Balagopal Unnikrishnan, Ariel Guerra Adames, Amin Adibi +10
While ethical arguments for fairness in healthcare AI are well-established, the economic and strategic value of inclusive design remains underexplored. This perspective introduces…
The Personality Illusion: Revealing Dissociation Between Self-Reports & Behavior in LLMs
Pengrui Han, Rafal Kocielnik, Peiyang Song +4
Personality traits have long been studied as predictors of human behavior. Recent advances in Large Language Models (LLMs) suggest similar patterns may emerge in artificial systems…
Prosocial Behavior Detection in Player Game Chat: From Aligning Human-AI Definitions to Efficient Annotation at Scale
Rafal Kocielnik, Min Kim, Penphob +5
Detecting prosociality in text--communication intended to affirm, support, or improve others' behavior--is a novel and increasingly important challenge for trust and safety systems…
Self-Anchored Attention Model for Sample-Efficient Classification of Prosocial Text Chat
Zhuofang Li, Rafal Kocielnik, Fereshteh Soltani +4
Millions of players engage daily in competitive online games, communicating through in-game chat. Prior research has focused on detecting relatively small volumes of toxic content…
Context-Aware Toxicity Detection in Multiplayer Games: Integrating Domain-Adaptive Pretraining and Match Metadata
Adrien Schurger-Foy, Rafal Dariusz Kocielnik, Caglar Gulcehre +1
The detrimental effects of toxicity in competitive online video games are widely acknowledged, prompting publishers to monitor player chat conversations. This is challenging due to…
Reinforcement Learning for Efficient Toxicity Detection in Competitive Online Video Games
Jacob Morrier, Rafal Kocielnik, R. Michael Alvarez
Online platforms take proactive measures to detect and address undesirable behavior, aiming to focus these resource-intensive efforts where such behavior is most prevalent. This ar…