4 papers
Evaluating Second-Order Bias of LLMs Through Epistemic Entitlement
Ramaravind Kommiya Mothilal, Terry Jingchen Zhang, Raiyan Ahmed +3
Evaluations of social bias in LLMs largely focus on whether models generate or imply biased content. However, as LLMs are increasingly used as judges of bias, they may exhibit soci…
Argument-Based Consistency in Toxicity Explanations of LLMs
Ramaravind Kommiya Mothilal, Joanna Roy, Syed Ishtiaque Ahmed +1
The discourse around toxicity and LLMs in NLP largely revolves around detection tasks. This work shifts the focus to evaluating LLMs' reasoning about toxicity - from their explanat…
Reasoning About Reasoning: Towards Informed and Reflective Use of LLM Reasoning in HCI
Ramaravind Kommiya Mothilal, Sally Zhang, Syed Ishtiaque Ahmed +1
Reasoning is a distinctive human-like characteristic attributed to LLMs in HCI due to their ability to simulate various human-level tasks. However, this work argues that the reason…
Talking About the Assumption in the Room
Ramaravind Kommiya Mothilal, Faisal M. Lalani, Syed Ishtiaque Ahmed +2
The reference to assumptions in how practitioners use or interact with machine learning (ML) systems is ubiquitous in HCI and responsible ML discourse. However, what remains unclea…